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ABSvTRACT 



The Navy computer-managed instruction system (Navy 
Ctfl)\ is a large, multi-si.te operation. Research findings show that it 
is yielding benefits^n terms of cost savings. During fiscal year 
1975 .savings of over ten million dollars yeve Realized, most ly due 
to course reductions ranging from 2U to 80 p.e^rcdnt and reductions m 
on-board students. It has been found that CHI tdainmg yields better 
end-6f-course performance' levels while maintaining more positive 
atti-tndes among students. Attritioii rates are lower with CMI, .and a - 
-continuation of this trend is pre diet e^^v-as the system ite expand-ed. 
Savings are projected in the potential for expanded capability and 
competitive procurement using the current system and l»ardware. 
Research has shown positive personnel attitudes associated with the 
integration of personnel and operational procedures using CMI. It is 
suggested that performance and cost benefits qualify Navy CMI for 
expansion. (Author/CH) 
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. ABSTRACT, [ ' , . ' 

Tlie Navy Oil system repre^nts tlie most foutstanding large cornputer 
based, individualized instructional system developpd to d^te. There, 
arc several" reasoi-is for titis ir,portant acliieyement . First and foremost, 
tliere has been and continues to be exem^ilary- training effectiveness 
witliin tlie system. The logistic adiievcment of a computer supporting 
in excess of 3,000 students in multi- sites ^represents a first in this 
fielCd.' A more dramatic adiievement is tlie cost beneficial outcome -- 
a savings of $10 .2 nullion during' FY .75 , a savings rarely found in tlie. - 
"initial life cycle of a training system. Finally, tlie Tkvy CII system 
has enabled institutional integration of Navy teclinical training 
iito jcommortvpractices and styles wliile acliieVing its ovm unique 

benefits. \ . \^ ,1 _ ^.r^ 

Tne develoWient perf ormanct\ data yielded to date by the Navy QH 
system provides di strong quantitative argu,\ent on its behalf. First 
the an system his yielded signific'ant course reductions ,ran^;Lng from 
24 to' 80 percen| ulth a mean of 48.6 percent. This has yielded a $10.1 
i;iillion saving ilji student salaries. Second,' tlie effective reduction 
in on -board students lias allowed for an associated reduction of 23 
percent in instructional/support.^ personnel \vliij:li has yielded savings 
of $1.7 million. Tjiird, the CH training ^approadi yields significantly 
better end-of -course performance levels u'liild the attitudes of students- 
tend to be more positive. .In turii, the QII system has significantly 
lowered the attrition rate, to somev/here' bet.veen 4.5 and 11.1 percent. 

Tliis kas yielded approxij^iately $550 ,000 in savings for FY 75 ; tliis 

/ 



should increase as the system is e^anded: Finally, the OMBpiiter^ 
Uipleroentatian, as currently Operating and in terms- of the aeqi^^ition 
of the Honeywell system for expansion, Has yielded savings in both the 
competitive yrocurement it^lf and the potential -for expanded capa- 
bility. / . i 

In terms of the institutional training processes, the CMI system 
has effectively intfegrated both personnel and operational procedures. . 
For example, the learning center supervisors and ISD personnel , »*io 
are highly cormnitted to CM's implementation and operation, perceive 
its approach to individualizing the training process as most success- 
fill: The attitudes of the personnel involved are'Mghly positive, 
thereby greatly benefiting the system's effectdve implementation and 

success . \ . „ ' • ' 

The Navy system also represents an outstanding exarnple of how an 
R5D activity culminated , in fruition of an actual training operation. 
'The research climate, shared civilian arid uniformed personnel,, a 
commitment to sound training desi-gn, and an adaptive? approach to CMI 
systems goals undoubtedly allowed it to move from the R5D phase to 
fiilly operational status in less than a decade .-^ Conpared to the 
University of Illinois^ PLATO system and the Air Force Advanced InstruQ 
tional System, Navy CMI is the largest and seems the natural candidate 
for both expansion and further elaboration in the future. While each 
of these three computer based systems has its own unique purposes, 
goals, and implementation characteristics , the Na\Qf^ system is 
yielding perfomance aid cost benefits that are especially attractive, 
in the mid 'TO's. . - . 



The future of the Navy CMI system is already designed. It sKall 
grow to support 17,000 students by 1980. During the"course>^f this 

' study, new enhancements have been identified. Instructional strategies 
and instructional' system development activities can give a sensible 

v,x«- infusion of proven m prototypes which would be highly cost bene- • 
- f icial dyirm the coming years, l^us , in a sense, this monograph ends • 
where if begins; that 'is, in a>alization that research and develop- 
ment can again contribute to this outstiinding^system which fortunately 
has been designed to infuse^iew ideas and concepts while maintaining 
its high cost beneficial mpact. 
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Tm NAVY CMI SYSTEM.: OmREm* STATUS AND POTENEIAL 

CHAPTER 1 . Introduction to the Navy's GNU System ' 

During the past^half decade Navy technical training, especially 
within its air am, has developed the largest computer based training 
system in the vrorld. The basic purpose of this document is to provide 
a description of this OH system to include its history and its future 
potential. A report has not been^ forthcoming due' to *the documentation 
concerh^g the Navy QAl system being enfcedded in various official 
reports. The purpose of this monograph is to provide a coherent ^ 
description as reflected by ^ following objectives. 

study Objectives 

1. To describe the current- status of the Navy Ofl sys-tem and to 
relate this to its R§D origins , its current climate , and its likely^ . 
future evolution . The primary purpose is to describe the system, past, 
present, ^d future. This is considered crucial so that individuals 
can see ttiaf. the system is growing not only in terms of numbers of 
con^uter coinxjneAts , but in terms of training sophistication and 
complexity.* More Jnportantly , the Navy CMI has had a climate, both 

R and D and operational, that .has facilitated it,s adaptation and 
positive accomplishments. ^ 

2. Tp identify CMI training performance oautcomes, as well as its 
cost benefit relationships for Navy Technical 'Training . To adequately 
assess a technologically based training system, one has to consider its 



training perfomance outcomes — learning rates , attrition rates , and 
general outcome perfoxmance levels. These, in turn, shall be related 
to the costs of the systan, especially as compared with' such alterna- 
tives as ^conventional instruction. In order to broaden the base for^^ 
viewing t^ie Navy CMI system, the purposes and operations of the Air 
Force Advanced Instructional System, Lowry Air Force Base and the 
PLATO GAI system at the University of Illinois, Chairpaign , Illinois 
shall be examined. The purpose of these comparisons is not ^o docu- 
ment that the Navy CMI system is the most co^-effective , since each 
of .them has different types, of goals, but rather to provide the reader 
idth some l Afivv standing of how the Navy CMI" system fits into the 
growing world of technologically based training systems, ^ 

3 . To identify future Navy ^training alternatives vfaich can be: 
integrated within the CMI sysX^sA . As indicated, the Navy is committed 
to allowing for bcth a growth in size of the'^ system and, more impor- 
tantly, an increase in its sophistication and complexity where sppro- 
priate. Appropriateness shall be judged primarily in terms of cost 
benefit relationships; thus, the Navy CMI system rejects no specific 
training styategem but rather views them all as conqjonents to be J\ ^ 
incorporated in the futui^ for active pursuit of the most effective 
means of training its personnel. We turn now to a description of Navy 

Oil. ' i 



Navy CMI 

CMI has-been in th^ R5D process for several years. It is an ADP 
system vMch prescribes a course bf study individually tailored to each 
student with enphasis placed on frequent testing and evaluation. The 
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student in OH systems does not, however, interact directly vdth the 
system as frequently as he* would in other fully automated learning 
systems. Tte CMI system provides the student with lesson guides and 
assignments vMcli lead him througji individualized instructional modules 
whidnnay utilize several media ranging from prpgranmed instruction 
booklets to audio visual materials. Upon con?)letion of a learning* 
module the student is tested; the test is graded and the resxilts are 
evaluated by the system. A product of this evaluation is additional 
lesson guides/assignments based on the student's test knowledge at 
that point, the student receives only the informatidh necessary for 
satisfactory achievement at a prescribed level. - . 
\^ These 1CMI modules may be programmed at two or more levels of 
difficulty* Iji addition, instruction may be ad^ted to the individuid 
requirements of students by three other means, namely, remedial loops'^ v 
within a mbdule^i repeating a module if it is ndt learned ^adequately 
the first time, aiid branching within a module in response to student 
answers at certain check points. It should be noted also that ind^i^^ 
viduilized instruction permits each student to progress through the 
course as rapidly as he can. This characteristic of individual pacing 
provides excellent opportunities; for the use of ^centives to encourage 
rapid achievement of training objectives . 

At Jhis stage of development in the application of the conputer 
to Nav]^ Technical Training, gives promise of providing an economical 
and workable system. CMI is designed to provide cost effective educa- 
„tion to \ large nunber of trainees in the military environment where 
students train six to ^ight hours per day or multiples via double or 



triple shifts. Initially, eleven schools are being projected to . 
utilize CMI; ultimately, twenty-four schools shall be involved (ulti- 
mate|^xpansion only network limits) . Thesfl schools represent many 
varied disciplines but have , some similarities. The schools normally ^■ 
consist of multiple classrooms. 'At Navy Nfemphis , these classrooms are 
housed in one-story wood frame'^^ldings with approximately five class - 
rooms per building. An average classrocrf will siflpport approximately ^ 
eighty students (range is thirty to one hundred) ^ Within this context, 
classrooms vdll often be referred to as l€?aming centers and a ^student's 
work area as his carrel. While assigned to* his carrel, the stxident 
will perform various functions and assignments to include: reading 
texts, utilizing films and slides , jperforming lab « experiments , taking 
"tests, perfomdng vnritten assigpents, and ptlier learning procedures. 
During this learning process the student will normally have some 
questions that require instructor aid. He will-also have many tests 
and written assignments that must be evaluated and scored. The student 
will also have to be directed to other assignments based on his previous 
responses . Scares will then have to be recorded and composites 
canpiled. « 

CMI perfoms the functions of evaluating tests, scoring these 
tests, determining results, prescribing the next assignment, and 
recording and conyiling grades . (This is further e^^^^StS^ in 'Chapter 
4.) By putting these administrative and other tasks on the computer, 
the instructor is available to more students and has more time to aid 
each student. 

CMI System Objectives 

In more quantitative tAnns , the following objectives have 



been specified for the Navy CMI system: . (1) an average reduction in. 
the in-cotrfse time \>€ students by 30 percent; (2) an average reduction 
•"in instructional/support staff by. 20 percent; (3) an enhancement of 
end of course performance levels (this vdll be modest as there is only 
a small margin for possible inprovement) and (4) a reduction in course 
attrition (rates of failure) . > 

* 

structure of the Report : 

' Given this basic imderstanding of CMI, w© turn now to consider s\^- 
sequent chapters this report • Chapter 2 presents the history of the 
on system^ especially in terms of its phasing from a R5D effort into 
an operational effort. Chapter 3 presents the data conceming sttident 
perfoimance, especially in terms of progress rates, end of coui^se per- 
foTtnance rates, and associated attrition data, the prime objectives. 
Chapter 4 presents a description of CMI hardware and language — past, 
present, and future. In addition, the characteristic of the CMI language 
(software programs) shall be descHbed and critiqued. It should be 
, noted that a very large conputer procurement program hlas reached its 
conclusion v*iile this report is being prepared. Chapter 5 presents 
instructor functions in CMI , especially those relating to role^ and 
areas for possible additional contributions. Chi?)ter 6 shall relate 
to the ISD course conversion process and a^so^iated management problems. 
Chapter 7 shall identify the cost benefits of the Navy CMI system. 
This will be performed prinarily within the context of a comparison 
with conventional instruction, individualized managed instruction (a 
fonn of programmed ijistruction) , and CMI. Chapter 8 shall- present a 
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context £ramefv/ork for large tomputer -based leaming/^systems. Chapter 
9 shall identify possible fulture potentials and alternatives for Navy 
(M, especially those that ^ppear likely to iiicrease the cost benefi- 
cial effects. Chapter 10 sAall provide a summary of the most signifi- 
cant findings yielded by ^this study. ^ ^ 




QIAPTER 2 . tiistory' of the Navy 



Systan 



The purpose of this chapter is to concisely describe the evolution 
of the Navy OsH' system. The prijicipal finding mil -emphasize how the 
CMI systCTi ph^ed through the research and development phase on into 
advanced development, and ultimitely, into operati^ in less than a 
decade. To create, ijnplement, test, and evaluate a Vstem in less than 
a decade is an outstanding achieUment considering tlie engineering and 
other technological ^vironinentai constraints of DOD. It is primarily 
for tills purpose as^ well as for the edification of those who do not 
know whert Navy C^I came- from tli^t tliis chapter has been prepared. 



instruction research exenplified 



Early R^D EffoAs 

The historical antecedents to Navy Oil include: (1) the Teaching 
Machine and/Progranmed Instruction (PI) .^foveInents of the l^.te 1950 's 
and"early/l96.0's; (2)" t]ie early (1960 's) nulti-agency computer based 



by the ONR spohsored work at Stanford , 



Texasv Illinois , Pittsburg, MIT, Bolt , Baranek and Newman. and many 
other organizations and educational research centers ; (3) the establish- 
ment of the Navy Training Research Laboratory (NTRL) with a Branch 
Office at Ivlenphis-; (4) the mutually supporting interactions of ONR, 
NTRL, and the researcli staff of the Chief of Naval Air Tedmical 
Training, and (5) the establishment of the Advanced Development Objec- 
tive for Education and Training (F-4303X)-. 

/::: , ■ • 

The earliest precutsor of CM activities (1950 's) can be -traced to 
the contract siqjport for programmed instruction research by Drt- Glenn 
Bryan, Office of Naval Research , to Dr.. Douglas Mayo, Branch Chief for 
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the Naval Training. Re sear di Laboratory (NTRL) . This early prograinmed 
instruction researcli provided tlie conceptual framework as well as 
personn(?l' for tlie^xieA^lifpinent of Oil courses. During the early stages 
of tlie QII project one could hardly discriminate bet/;een a prograimed 
instruction course (PI) and a conputer nianaged instruction course (03). 
Oil moved beyond tliis in a very short period of time.^ ^ 
\^ Thep Office of Naval Researcli sponsored a training evaluational 
project at V/asiiington University, St. Ix)uis (Du Bois , et al , 1970) 
tliat provided a cliinate 'conducive for tlie development of tlie Oil ^ 
system by placing young scientists at HAS llcmplus. In a general sense, 
tlie efforts of Dr. Suppcs at Stanford lJniversit>^ ,in Rooking at large 
Student and data, based drill and practice systems ,\fas^xiinport ant . Also 
tlie work of Professor Glaser of tlic University of Pittsburgh in lookiiig 
at tlie integration of Oil witliin a pul^lic sdiool setting allowed for 
tlie early identification* of iiiajiy o:^ tlie testi^ig logistic requirements, 
j* yUtliougli supported indirectly by ^W.y tliese two were sponsored by 
civiliaii agencies. Finally, the ongoing CAI, Oil projects at Florida / 
State University, si:)onsored by Project Themis, Department of Defense, 
and moni.1^ored by ONR contributed iiv a concurrent effort to evolving 
the ef ficatiesr of many of tlie proposed computer based training altema- 
tives . \^iiLfe many other sources could be cited, it is eAadent tliat 
all of tJ^s^ e£foi*ts provide a climate for tlie training rfes.eardi and 
development qf a clear understanding as to how the first Oil system 
should be assembled'. Tliis was obviously a critical first step in tlie 
phasirfg^fjom R§D on into operation. 

^ • .... *-ii?7v, ■ ' 
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Advanced Development ^ r 

In 1966 , the Assistant Secretary of Defense for Manpower 5 Reserve 
allocated $70,000 to initiate the QII prpject. The focus of tjiis study 

r 

Was' on individualization and funds wre provided for both an ^in^jlemen- 
/tation as well as computer assisted instruction (CAI) explorations . ^ 
The CMI project, as directed by Drl Douglas Mayo, was initiated 
by the Chief of Naval Air Tedmical Training, Rear Admiral E. E. . 
C3iristianson, at NAS, Millington, Tennessee in July, 1967. It was 
approved by the Chief .of- Naval Air Training and 'the Chief of Naval 
Operations. Subsequently, tlie Navy's advanced development objectives 
(43-03^ personnel and training) providedl tlie preponderance of funds 
xthrough, the 'direction of its Chief of N^val Personnel for the M 
project. Thus, the QII project in its formal sense was initiated in. 
fiscal year i968 ^is a joint undertaking of the Qiief of Naval Air 
Technical Training and tlie Navy Training Research Laboratory Brandi 
: Office, Menphis.. It should be noted that Dr. Mayo held ^pointments 
in botli of these linits. - Such cross appointmerfts can be critical in 

tlie early stages of rapidly moving a system through R§D, adv^ced 

■ / ^ . ^ . _ 

development,. ^d on into operations. ^ f — TS; , f 



in the 
point 



B?y July, 1969, tlie development of the \CMI jmstructional materials 
Aviation Nfechanical Rmdamentals Sdiqol had progressed to the 



that it was feasible to extend :.the Qa\project to the Aviation 



• Farailipization Sdiool. This course was an orientation to Naval^ Aviation " 
taken by all trainees ordered to the Naval AirVrecIinical. Training 
Center^ ffempliis from the Recruit Training Commands. This amounted to 
an input of approximately 500 trainees per week . ^Ihe i)resent course 
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was tvvo vfleeks in length (originally six weeks) yielding an average time 
reductionnmder conditions of CMI of 2/3, or 67 per cent. 



Joint Institutional Development . ' ■ ^ 

As is common to training researdi, a joint institutional develop- 
ment \vas pursued, hftr. Bernard of IBM prepared the first concept paper 
that .lead toxa contract with State Technical Institute of f lemphis. 
The ffenphis State. Ur|iversity Computing Center developed tlie computer 
software tJiat si^ports tiie CMI system.- In so doing, 'tl^^^used an IlSM 



360, Ihdel 40.co]T^^ter ;dtli an IBM 278(T teminal. This^ teminfi is a 
relatively liigii spedd input-output device, ^vhidi can direct tlie learning 
activities of a much larger number of students tlian can be handled by a 
typical teletypewriter terminal . 

Under a separate contracf^ the. Bureau of Educational ;/Researdi and 
Seif^riceB' at Menphis State University siflported tlie project with reseakdi 
perta^ffiin^~tD media selection and \\at}i assistance in prep'aring and 
coding instructional CMI materials . ' ^ ^ 

The Uhiversity of Tennessee Biometric Computer ^fenter adapted a 
re.Jatiwly new CAI language, called Coursewiter I'll , to the CM 
requirements of the project before the langu^^e was released for 
general use. In addition, during the initial contract \\riLth the Univer- 
sity of Tennessee, several hours of tutorial CAI instructional material 
pertaining to the Navy 3-M system were developed and tested, Tliis work 
involved an IBM 360 Model 40 cpmputer \dtli a terrtiinal, IBM 1050V. Tliis 
terminal consists of a teletypewriter and lias a random access slide and 
audio tape capability. Subsequent to tlie initial QDntract, €he 
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_ J Biqinetric Computer Center continued to supplement tjie project in- house 
GAI capability, as needed, witli a renote terrninal located aboard the 
-Naval Air Station. Tlie findings from tliis researcli indicated tliat CAI 
should be delayed for ijnplementation witliin Off. Sucli early investi- 
-. gations allowed for greater competence in pursuing the test sheet 
oriented OlJ terminal (v^icli is a compromise CAI terminal) than that/ 
t^ically found in larger remote batch, teniiinals^in Oil . Thus , Dr/ ^ 
' Douglas Mayo and Dr. Larry Harding led tlie QH system into its dpcu- 
mentable state v^iicli provided enough evidence to allow for authoriza- 
tion and funding as an operational resource -for CNTT. -Dr. Kirk Jdrnson 
assumed the responsibility for basic reseatrdi within the project. It 
should also 'be noted that tlieir .researcli and tliat of their colleagues 
(Charles Tilly and Chief Petty Officer 0 'Neil) provided extensive' 
evidipnce concerning hovy appropriate 'QH re -designs could be pursued. 
Additionally, ^Ir. Robert Potts provided the first file structure design 
for the system. ^ 

CMI. Personnel 

Within^this project, the mix of civilian and uniformed personnel 
types was quite facilitating. First, tliere were the civilian training 
researchers as .represented by Dr. Douglas l4yo, Dr. Larry Harding 
and Dr. Kirk Johnson, Dr. Johnson fostered the researcli to action,, 
interface. Dr( Harding, Stuart Carson, and Phi His Salop were respon- 
sible for the OH developmental work (Naval Training Research 
Laboratory). Perhap^^, more, .pportantly, Mr. Charles Tilly, ,M?. Robert 
Potts, and associates, civil servants within the computer «upport world 
of the Navy, assisted in the ultimate iji^lementation of both the 

* 
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advanced development and the operational computeri system. In reference 

1 I. 

to the conversion of instructional material, tlier^ were both civilians, 
primarily from Menphis State Uhivei^ity, and Navy personnel. Borne of 
tliese Navy ISD specialists continue to contribute ^t this time ; con- 
^ tinuity of this kind is invaluable. 



Operational System 

^ " CMTEarmA submitted aA interim report on 26 October 1970 seeking 
approval of the OMI system' al. an operational el;ement within Navy 
training. The Quef of NavaE Air Training and CNET approved .this 
action and advocated the CMI |ystem^to higher command levels. Captain- 
Bruce Stone and Dr. Worth SGa|iland sponsored the computer based cost 
effectiveness simulation (CMIr became cost beneficial after N>-1300) and 
formulated the basic rationale for tlie decision to go ^pjgrational . 
Tferl\wk# approved by tiie QnIO on 5 February 1971. The cost justification 
^ ' data clearly indicated the advanced evolution of this OH system. (See » 
□lapter 7.) A brief summary of this data should convince the reader. 

Like industry, but unlike universities and public sdiools , the 
armed forces pay tiieir students to ^undergo training, tliaf is , they 
continue to pay the students' salaries wliile they are in training 
"status. .Jime saved tlirough more efficient training procedures permits 
an increase in productive manpaver' in operating units , or reduced 
overall manpo\srer requirements tdthout reducing tlie number of personnel 
in oi^rational units. Certain facilit/ and material costs often 
accompany reduction in ^course lengths, but only personnel costs are 
included in tlie present figures. Tlie following two paragraphs describe 
the cost avoidance feature as envisioned at tliat time. 
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Tlie reduction in average cx)urse lengtlx that is expected, for 
exmttple, for the Aviation Fundamental Course is, for example , from' six 
\veeks to ti^- weeks and for tlie ^fecllanical Fundamentals Course , from - 
tliree vgeeks to two vfeeks. (These t\^o courses are currently combined 
into one, AFAM.) This results in annual savings of SOQ.man years and 
l'/5 man years, respectively. At an average salary for pay grade:E-3 ' 
of $3520 per year, tKis' comes to $2,376 ,000. To tliis figure -should be 
added the salard^es of 32 insATiCtors vdio will^nof be needed in" the 
shortened courses. At $7,460/, tlie average salary of pay grade" E-8, 
this saving comes to $238,720. Summing 'the student saving and tlie 
instructor savings, \^ haxfe a total gros 5 saving of $2^1514 ,720 per year. 

Tlie hardware required for the Q^II sjfetem should cost about $^36 ,000 



per year if leased an(l installed at the Naval Air Station , Ifemphis 
(see Chapter 4). Computer operating personnel costs would approximate 
$130,500 per year. IVhen the hardware and personnel costs are added to, 
the $12,000 per year ^br supplies and miscellaneous expenses is added 
to the total cost comes to $478, 5p0 per year.. Subtracting rtliis cost 
from the gross savings of $2,614,720, results in an estimated net annual 
saving of something over $2,000 ,000. Tlie above figures were utilized 
for the initial justification to tlie CNO. 

Computer hardware and operating personnel requirements for CMI 
and other ADP applications were set forth in tlie CNATEQITRA Command/ 
Managenent Information System plan submitted to the Chief of Naval Air 
Training on 28 May '1970 and subsequently forwax;ded to tlie Chief of 
Naval Operations. This plan included installation of ADP equippienr at 
the Naval Air Station, Ifenphis on 1 July 19.75, capable of handling CMI 
requirements'. * 
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However, subsequent e££orts\to gain resource (fiscal) si^port fpr 
operationalizing and expanding tlie 'sy^tert \>rere unsuccessful at the 
CNO/OSD levels because of tlie mis concept ions at tiiose levels of the . 
applications of computers to training. a succeeding occasion for 
the presentation of arguments in support oE ADP- equipment for tlie CMI 
pperationalization-, the CNET spojisor, Dr . Woft)i Scanland, presented to 
the CNO/OSD reviewers' tlie position that tlie management of liigh student 
density individualized instruction created *^uch lat-ge rec^uirements for , 
infomation and data processing that the only way in i^iidi sudi ' 
individualization was possible was tlirough the- application o£ modem - 
ADP tediniques. Couched in tliese term^^, the requirement for y®P 
support could be defended in the , same context as other ADP recfijiirements , 
ratlier tlian in the context of an iiistr^ctional mediating device. .The^ 
resources were approved \>dtli no further delay's, and the long process- 
of ADP equipment acquisition \Vas alile to commence . ' .\ 

Finally, the cpurse conversion took place in tlxe Basic Electronic 
and Electricity Sdidbl, Navy personnel' along witli Mr. Charles Tilly, 
provided for a significant re- design in the teminal equipment for the. 
learning centers. TMs use of sheet oriented test op-scanning devices, 
a CPX, and a low cost inpact printer has contributed invaluably to tlie. 
smootli operation and growtli of this system as well as its enhanced 
cost effectiveness. 

As originally <lesighed by tlie R and D group, a remote batch terminal 
Qiigh speed card reader aijd line printer) was^ utilized by having human 
messengers from -each tenter. The cost,, time delays, and tendency for 
queuing i^om tliisxx)nfiguration was unsatisfactory, llie Tilly proposed 
terminal configuration resolved tliese problems and enlianced the system , 
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response time by smoothing, tlie input distribution. 



Conclusioi^ *■ - \ 

1. The' Navy Gil system is an outstanding example of how ttsiiriing 
research and development can be brought to fruition in actual training 
operations in less than a decade. This was immeasurably aided by the 
research climiate/ shared personnel; rejection of unproven .training 

|alternatives , and a CQinmitiiient to -the Qll systei]f goals. . ^ 

2. » JTie ndx' of; civilian researdi psydiologists (primary manager^) , 
Navy personnel / aiici university personnel provided the ricliness. 
Critical mass ^so .necessary' to tlie design, validation and operation of 

a system. . 

I . ^ : ■ ' ' ■ ^ . 

3. The an research effort yield tliree immediate outcomes^ 
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(I) a prbtc)^6 ail system: that required furtlier design, (2) a delay 
in CAI implementation mtil!qost justifial^le, and (3) a significant 
reduction in course training tiine tliat jiia^ifted moving into operaticMis, 
Obviously, appropriate researdi can be initiated at any time. 




CHAPTER 3. Student Perfoimance In Navy CMI 

Ai^ indicated in the introduction to this monograph, there are 
two ppicipal objectives for the Nayy CMI system. These are (1) a 30 
percent reduction- in training time for students and (2) a 20 percent 
reduction in instructor/support personnel requirements . As corollary 
objectives, there is the intent (3) to improve the level of end of 
course mastery and for (4) a reduction in course attrition or student 
failures. Each of these objectives shall be reviewed in- terms of , ^' 
current available data. The^e data were gathered during the ,last 30 
days from the CMI. system as well as from prior studies vMch wer^ 
coiH)leted within the last 12 months. 

J '4 

TiiTBe in Course . „ , * 

The data of three CM courses currently in operation came from 
three sources: (1) currently enrolled. students (early April, 1975), 

(2) cuimilative data from the July- December, 1974 period, andj:^3j^^- 
parative data on an experimental study at Great ^akes ("Formative 
Evaluation of An Experimental BE/E Program," Fishbume and Mims, March, 
1975). There ai« three courses that contribute data: (1) Basic Elec- 
tronics and Electricity (BE/E), (2) Aviation Fundamentals (AFUN), and 

(3) Aviation Mechanics Jet (ADJ) . « (Tables and Figures shall indicate 
these groups appropriately.) ' ' 

For the three courses^ Figures 3.1, 3.2, and 3.3 present a com- 
parison of the CMI mean accunulative time per module as contrasted with 
thegprior fixed conventic^al inst?uction time. (The tifne for the CI 
course units has been smix>thed as to minor unit to unit variations.) 
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COURSE SEGMENTb- , • 
Mean Learniitg Comple'tion^'i^me^ f or , Ai)J 

Figure 3.2 
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Figure 3.3 
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• As indicated in Figu^ 3.1, student progress time through the CM 
BE/E course is significantly and progressively less than that pf the 
conventionally taught BE/E course. CIh6 computer module numbe^rs are • 
not exactly equivalent to the content units due i^to segmentation of 
laf^e content units and large exams.) Notice that the final segment of 
the OH BE/E course is conipleteci at 159.3 hours, while the conventional 
instruction is COTpleted at 210 hours. While the qbsence of identical 
instructional segments prevents a module by module comparisan of the 
two versions of the BE/E course", the projected linear relationship of 
the conventional instructiort serves to illustrate the contrast with CMI 
Likewise, this relationship in much greater contrast is shown lor 
representative tracks of the ADJ and AFJN courses in Figures 3.2 and 
3.3. This illMStrates that there shall be a range of course time 
reductions that center somevtfiere between 35 and 4S percent • . 

For more detailed vd thin- course outcomes , a contrast of a textual 
(reading) only treatment, as opposed to an ^;audio- visual training 
treatinent vdthin the AFUN course is also pres^ted in* Figure 3.3. As 
is coititohly found, the reading course version is ^lightly faster. For 
the bri^ter students, textual versions of a GMI course are undoi±>tedly 
the best. Audio visuals have a place for the lower category learner 
-and for those performance skills requiring Adsual demonstration. 

Table 3.1 presents these percentage time savings in quantitative 
form. 9i4iously, a statistical test is not required to confim the 
significant superiority of the CMI groip. 

^^^iven the nature of v the samples (all course sanples exceed 3000 
students) and the consistency across courses, CMI is obviously 
achieving its plahne4 objective of a 30 percent course reduction. 
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Table 3.1 



Course 



BE/E 

AFUN- 

ADJ 

Mean Total 



Mean Course Time -Savings Via CMI 
— 



Conventianal 
Instruction 

I Time 
— ; ■ 



210 hrs. 
180 hrs. 
198 hrs. 
196 hrs. 



Oil Time 



159 .3 hrs . 
35.45 hrs. 
118.2 hrs. 



104.3 hrs. 



■t: 



Percent 
Reduction 



24.1% 
80 .31% 
31.0% 



46.8% 



r 



Table 3.2 

NIMBER OF INSTRUCTOR BILLETS FOR FY 72 - 74 



Billet Description 


FY 72 


FY" 73 


FY 74 




AEJ - NATTC Menphis 


'69 








c Authorized 


64 (7.3%) 


55 (14.1%) 




On-board 


66 


66 (0%) 


55 (16.7%) 




AFUN - NATTC Menphis 










Authorized 


116 


, 98 (15.5%) 


77 (21.4%) 


1 


On-board 


. 134 


96 (28.4%) 


92 (4.0%) 





♦Figures iii parentheses indicate percent reductim from previous year. 
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^ InstTUCtor/Suppbrt Personnel Requirgnents . 

Table 3.2 presents the authorized and on- board nuiribers of instructor 
billets at two NATTC Menipfus schools transitioning into the CM system 
fromi^y 72 - FY 74, It is apparent that .significant reduction occurs 
as (M pro^sses in theSe schools. Starting vdth th/baseline non- 
conputer managed AEJ course in FY 72, Table 3,2 reveals a 7.3 percent 
authorized reduction for FY 73 and a 14,1 percent authorized reduction 

""for FY 74. This total authorized reduction of 21.4 percent is approxi- 
mated by the 16.7 percent on-board reduction actually achieved. Like- 
vds«, the AFUN course experienced a 15,5 percent authorized instructor 

'billet reduction in FY 73 and a 21,4 percent reduction in FY 74, The 
total on-board reduction actually achieved for this, period was 32,4 

percent. ^ ' ' . 

Due to the creation of the BE/E course from portions of previous 
school courses, transitional figures showing p^gressive billet 
reductions are not conpletely available. The BE/E course;did, however, , 
achieve a 16.1 percent reduction dp on-board instructor billets for 
FY" 74. For the two year interval, this probably approximates 20 per • 
cent instructor course recluction. This can be observationally verified 
in that existing IMI courses require six instructors for a section of 
stXKients (N =■ 80) v*iile QjH functions with only three instructors. In 
Sane cases, tine^rplus instructors have been assigned to future CM 
conversion functions rather than being transferred off-base. 

In terms of the second objectfve , there is a 23 percent overall 
reduction in instructor personnel. As to be discussed in Chapter 7 ,^ 
this yields a significant reduction in support personnel. It should 
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be noted that a return to conventional instruction would 
jiKTease the budget for staff by 35 percent, a highly sigiif: 
factor. 



ijimediately 
leant 




- V 



students IMder Instruction 



Fi^re 3.4 depicts the weekly load of students under 



instruction 



from July 1974 through January 1975 (excluding Christmas vieek) for 
the BE/E, ADJ, ^ AFUN CMI courses. While the growth fnm a total 
student load of 1698 in ^e first week to 2902 in the twenty- eighth 
week is ijupies^i^^^^j^the fluctuations from week to week aret alsc^f 
significance. Gb^dousl^ the system is extremely flexible in its 
capability. The transftion from a student load of 2160 during the 
twenty-sixth week to 2843 in the twerity- seventh week is particularly 
in5)ressive. There are no other exanple^ of a computer based training 
system that allows for expansion factors like this one . 'The logistic 
prxjblems (e.g., student registration) an^ orientation reqijirements 
have' tended to constrain these con^uter systems. This evidence, 
supports the observation that Oil's most inrnediate beneficial contri- 
bution is in school house administrative requirements. 



Levels of Learning; Mastery . , 

While extOTsive comparative data are unavailable due to the absence 
of detailed records from the conventional courses, the Great Lakes Study 
(Fishbume and Mims, 1975) plus FY 75 accumulative BE/E dat^ can be , 
reviewed and composed. Table 3.3 presents results for the perfoimance 
level on the first attempt at the^ final comprehensive tesi (all students 
are ultimately remediated to the 100 percent level) by conventional 
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u- Figure 3.4 

Total Student Load for 
BE/e; ADJ, and AJWN GMI Courses 
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29S0 
2900" 
EBS0 
2P00 
2750 
2700 ■ 
feES0/^, 




July 
1974 



i.sff PI T ^- " "^ — ^ ^ ^ ^ ■ . N PI PI w n P i r^w n 



Week 



January 
1975 



ERIC 



Table 3.3 

i 



Mean BE/E Mastery Performance Levels and 
Ckanpletion Times For Conventional. Instruction , 
IMI, and QH Groups 



Instnictional 
Type 


N . 


tteari Scores i^r 
1st Pass on Final 
Canprehensive Test* 


Mean 
Gonpletion 
Times (Hours) 


CI 


50 


79.30 


214 .168 


IMI 


50 


77.69 


163.^62 


on 


1556 


82.73 


152ii7* 




Module No . 


# tff Items . 


, x<(Textua 


1 


39. ' 


2.54 


2 


31 


1.71 


3 

> 


^> 16' 


2. 12 


4 

« 


31 ^ 


1.60 


5 


48 


.98 




39 


1.04 


7 


47 


5.16 


8 


.27 


.38 


9 


- 22 




10 


41 


5.33 


11 


50 


.84 


12 


25 


1.51 


13 




1.45 


14> 


25 


.72 


15 


39 


2.51 


16 




3.29 



I 



Table 3.' 

Mean Error Rates on Module Tests' for AFUN Course 

% . X (AV)' % 
3.74 
1.69 
2. 37 
^ 1.64 
1.19 
[ 1.33 
4.84 
.35 
2.54 
4.76 
.92 
1.53 
1.74 

>^ .44 
2.83 
3.47 
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instruction, ihstiuctor managed instruction (IMI is a form of progjranmed 
instruction), and CMI. As presented, the CMI group is superior in 
leaining perfomance levels (only students viho were qualified for "A" 
School are included) plus requiring less learhing^time. 

In turn, Table 3.4 presents mean error peixrentages for module tests 
in AFUN. These mean error rates indicate, that CMI is jexceeding a 90 , 
p^cent performance rate. Considering tiiese data plus the conparative 
instructional approach data, CMI consistently yields superior learning 
as demonstrated^ the mastery tests. . - / 



Course Attrition 

Unfortunately , CMI as cfpposed to CI training attrition da^ta is 
unavailable in a cxwparable form. Siapce NAS Meirphis is the only user 
of CMI for the BE/E. course up to Jgnuary, 1975, the. attrition data for 
pre last half of Fi 74 compare Instructor Managed Instruction (IMI) ^ 
at San Diego and Great Lakes vdth CMI treatments at MCT^)his. Table 3.5 
presents the attrition data by location, attrition type (/kcaderaic vs. 
non-academic), and mentaT^groups , I - IV. , Obviously, the CMI yields 



hig^ily signifidftt lower attrition rates (for these sample sizes, 
a difference of one percent is significant beyond^the P less than .01 
level). The percentage of reduction achieved by CMI ranges from 4% 5 
percent to 11,1 percent. These reductions are both statistically 
significant and, more importantly, systan significant in cost plus 
manpower terms. 

General Observations \ 

In the development of instructional technology and conputer based 



Table 3.5 - 

BE/E SCHOOL ATTRITION 
(Jan-Jul 1974) 





Great Lakes 
IMI 


Menphis 
Off 


San Diego 
IMI 


V 

Percent, Flow (Sanqple Size) 


2989 


3210 


2579 


OVERALL MTRITICN 


17.9* 


6.8* . 


11.3* 


School Academic Attrition 


9.6 


2.2 


' 9.8 


^ool Nbn-Acaderaic Attrition 


8.3 




1.5 


Sdiool MG I and II Attrition 


5.7 


T C 


3.8 


School MG III Attrition 


°9.9 


2.4 


7.2 


School M; IV Attrition «, 






.3 


School NG Unknown Attrition 

A 


1.1 


.5 


•8 



*A11 atti^tion statistics are perf:enta(fees of the total stxident flow. 



systewB, there are no, other examples like the Navy CMI Systein in terms 
of initial achievement of system objectives. Obviously, the cost 
benefit ijnplications are highly positive.^ To return to a convehticmal 
instruction approach would be exceedingly costly to the Navy. >€iven 
this two-year performance period in vdiich all objectives haye been 
attained or exceeded, future expansion could be even more cost bene^ 
ficial to the Navy. ' \f 



Conclugians 

1. on yields significant reduction in cdiirse instructional 
length, varying from a 24 percent to 80 percent reduction. The CMI 
goal Of a 30 percent reducjtion is obviously attainable. Even greater- 
savings can be achieved given new training alternatives for CMI. 

2 . TTie fO percent redyction in ins tractor /support personnel f 
is being achieved" the actiial reduction being 23' percent. . . 

'3. Ofl yields significantly better end-of- course pej^o^^ 
levels , and the attitudes of the students tend to be !isre positive 

(see Chapter 5). " - ' Xl . ' ' 

4. on ymlds significantly lower^trition rates- approximately 

4.5 to 11.1 percent in magnitude. | 

5. The Navy CMI system is unique in the early achievement o£ its 
objectives; in facty the achiJWent^, tend to\exceftd the objectives. 
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CHAPTER 4 . Navy Qg Hardware «nd Software Language . 

This description shall cover both the current and future Cprodire- 
nmt ccnpl^ted) CMI configuration, the description of the coming 
system will refl^ect both the hatdwaiie and software to be installed at 
MAS 9 Mnpphis. In. turn/ using a con^ination of contractor and Navy, 
personnel, the CNI -language has and shall have to undergo certain 
fimdamental dianges both for conversion and efficiency reasons* All 
future nodifications shall be made by Navy personnel. Hie focus of 
this chapter shall stress technical aspects of the current and future 
systems idth somd ;insights into the plaiming/procurem^^m ^ 
(The ncn-technicail reader may wish to branch iiniiiil jlitp1))ii|Mf ) - 



4,1 Present Hardware Configuration 

Jhe new Navy cooq>uter system will replace 1^1 ccn^ter and conmi- 
nication capabilities, currently under service contract thrptig^ Mnphis 
State Uhiversity. This includes the central computer , an XDS Sigma 9, 
located on the MSU canqnis and terminal equipment I both the administra- 



tive RBTs and classroom dusters tailed at ^^TTC9 Men|>hiSy NTG, 
Great Lakes, and NTC, San Diego. The equipment to be replaced is 
documented in detail. as follows: 
Central Conauter 
' 1-8610E Sigma 9 System w/128K words 
I78665E Port E3q>ansian 

1- 8675 MIGP Oumnel B 

2- 8671 Four Byte Interface 
1-7012 Keyboard/Printer 



/ 



ERIC 



1 



1-7122 Card Reader, 400 CPM 

I; 

1-723J. RAD Controller 
1-7232 RAD Ihit, 6.2 MB 
1-8680 High Speed RAD lOP 

1- 7212 RAD Ihit, 5.3 MB 

2- 7240 Disc Controtler 
2-7241 Extended Width Interface 

2- 724B Disk Storage, Four-Spindle, 200 MB 
1-7315 Tap Controller ♦ 1 Iirive 
1-7315, Magnetic Tape l*iit 

1- 7446 Line Printer, ISOa LPM 
V 1-7630 Conmunications Controller Plus 8 Lines 

3- 7631 Eight- Line Expansion liiits 
5-7601 Data Set Controllers 

^ 5-7602 Full Duplex Features 

Terminal EqulpMent 

(1) ClassTooM Clusters ^ 
^4^QPSCAN 17 Scanners 

24-(^ 713-10 CRTs ^ 

24 -CDC 713-120 Printers ' 

(2) AAdnistrtive Terminals 
3-DATA 100 Model S4 RBTs 

2- IBM Systan 7/ IBM 3780 Printer/Card Reader siibsystems 
Data Entry TCTiinals 

2-CDC 713 CRT 
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Commnication 

24-300 BAUD dedicated, unswitched full dtqilex lines vdth Bell Telephone 
113 modems a€ each end of the line. 

5-4800 BAUD, dedicated, unswitched, C-2 conditioned, full duplex lines 
with 4800 BAUD Bell Telephone modems at the ends . Frequency- 
division multiplexors are used for the line €b NTC Great Lakes'^ 
and NTC San Diefp. 

It .should be noted that the ratio of students to this equipnent 
'is approximately 70 to 1. It can, in fact, grow to 80 to 1 as repre- 

• sented by the classroom dusters. Such a student to equipment ratio 
is most advantageous and is undoubtedly the clearest indicator of thct 
type of system being utilized by the Navy . 

4.2 New NaviAMI Configurat^ion 

To facilitate the high level of system effectiveness required by 
the CMI system, connputer manufacturers participating in the procurement 
were required to design a configuration vJiich contained, as a mininun, 
two central. processing units for reliability reasons • Both processors 
are individually capable of processing all application programs and 
able to address all storage devices , remote terminals , input/output 
units, other peripheral devices, and modules of central memory. 
Availability rate (ysp time) for the central site system was set at 95 

* percent of the scheduled time in a given month. This performance 
requirement was established not only for the 30 day acceptance period, 
but Honeywell , the successful vendor, must maintain tl^s level of per- 
foxmance throughout the systems life (6 years)* Failure to meet this 
availability reqiaremebt will entitle the Government to collect credits 




/ 

for systens downtime (one percent of the installed systess total 
nonthly charge for each percent over five percent) . 

The new OH system is 50 configured ^t no device will, v*ien 
down, render the central system incapable of siqjporting the interactive ss^^ 
process between the student and the computer for a period longer than 
10 minutes. The system will, at all tines, include a minimum of two 
disk (iiiinediate access storage) controllers e^ of vdiich is accessible 
by both processors . This gives the capability to access any disk file 
through alternate paths in the event any one of the^ controllers 
becomes inoperative. 

Presented in Table 4-1 is the central site configuration scheduled 
for installation at -MAS, Mehqihis. Note the dual CPU^s system control- 
lers, iremory, input/output multiplexors, disk controllers, system 
consoles, and datanet coraramication processors . Configurations with 
) more than one of any module not only increase througjiput ,* but also 
/ provide built-in backup. And if a portion of the system does go down, 
Honeywell's General Coniprehensivp^Operating S^tem (GCOS) allows the 
^ user to quickly reconfigure the system to work around the failed 

module or peripheral and keep serving the Navy schools. 

Tlie front -end network processors (datanets) see to it that the 
coramunications network ronains available also. Their own stand-alone 
operating systems not only give better service to the users by removing 
the overhead of data manipulation from the /icentral system but protect 
against system downtime by continuing to perform many tasks even vdien 
the central system is down (a form of centralized and local processing 
* combined). If the ^tem, however, should go down, automatic restart 
■ • ^ . W 
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; , Table 4-1 

Initial Central Site Equipment lAst 



Model No. Description 

CPS 6202 Central Processor System, 131 ,072 

36-bit words manory 

MSP 0600 IAS Disk Processors 

MSU 0400 Removable IAS Units , 117 million 
° characters each 

MTP 0600 Magnetic Tape Processor 

MITJ 0500 Nine-Track, 800/1600 CPI Tape IJhit 

MTU 0400 Seven-Track, S56/800 CPI Tape Unit 

URP 0601 Peripheral Controller 

PRU 1200 ' Printer, 1200 LPM 

CRU 1050 Card Recorder, 1050 CPM ' 

CPZ 0201 Card Punch, 300 CPM ^ 

CSU 6001 Console 

DCP 6632 Datanet Front -end Processor 
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aand recovery features guaxd against lost information. The central ' 
host system is presented schematically in Figure 4- 1 • 

At the remote training sites, message concentrators vdll be 
installed to concentrate classroom cluster and administrative terminal 
traffic/ The message concentrator proposed by Honeywell is a multi- 
function system. It is configured to handle the combined functions of 
the administrative terminal and the remote concentrator concurrently. 
It will receive and validate messages and store and forward message 
traffic betiveen the adninistrative terminals, classroom clusters, and 
the Ntenqjhis host computer. This ncssage concentrator provides for 
ccraplete accountability for traffic in antTout of the ^concentrator , 
retransmission and alternate routing of input/output transactions, and 
retrieval of previously received messages. In the event of central 
processors failure, or failure of the comnunication lines to the owitral 
site, the concentrator will be capable of receiving messages from the 
5ilassroom clusters and the administrative terminal^without intemption 
or delay. The message concentrators will be linked to the host 
computer over dedicated 9600 bps lines, with 4800 bps dial-up as 
backup. As shown in Figure 4-2 the remote sites have also been con- 
figured to provide maximum availability. Dual concentrators are 
connected via switching units that allow all classroom clusters to be 
switched to another concentrator in the case of a concentrator failure. 
All administrative terminals have a primary and alternate route to 
^parate concentrators . Canmunication paths on the host side of the 
concentrators are configured so that they are always provided a rou^te 
to the host oomputer". 
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^'^^^JS^ all terrainal ^dusters and administrative terminals 
are interfaced directly to the host conqDuter via common carrier 
fMnamication lines. 

To give a portrait of the CMI system size vdthin Navy training 
(See Chapter 7 for further details), Table 4-2 presents the classroom 
teiminal schedule by fiscal years , and T^1^4-3 presents the adminis- 
trative tetminal schedule. One can observe that the significant build- 
up will take place by FY 78. 

Another facet to the system availability is the engineering tech- 
nology employed in the design, manufacturing, and maintenance program 
of the system put together to meet the stringent requiremwits of CMI. 
With Honeywell •s Series 60 conputer having the latest advanqed tech- 
nology incorporated into its design concepts, it will ensure Mgh 
uptijne. For example, error detection and correction logic in main 
memory provides a high degree of data accuracy. While processing is 
being done, online testing programs will check all portions of the 
system. Automatic error analysis and logging pibgrams will provide 
fast diagnostic reports, thus pinpointing potential trouble spots and 
avoiding unnecei^sary interrtptions . On-site maintenance support will 
also be available at the Memphis site throughout the training day. 
These and other items all add to the reliability neojded to success- 
fully iJl^>lement CMI. In short, the Training Coranand can be assured of 
dependable perfoimance because the CMI ccnnputer system was designed 
for roaxiimim availability. 

The Series 60 computer acquired from Honeywell for the continued 
support of CMI is a Series 60 large scale Level 66 computer com>onents. 



'5 



Table 4-2 



aassroom Cluster* 

^ - 

Installation Schedule 





76 


77 


W 

78 


79 


80 


NATTC Memphis 


29 


► 52 


' 73 


73 


73 


NTC San Diego 


8 


27 


37 


37 


37 


NTC Great takes 


28 


66 


79 


84 


84 


mc Orlando 




16 


16 


16 


16 


OTTCI^ridian 








21 


21 


Total 


^65 


161 


205 


.231 


231 



♦Each classroom duster consists of one Honeywell Model^ No. TN 
1200 12C CPS Keyboard/Printer and an OPSCAN Table Top Scanner (300 
sheets per minute) . 
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Table 4-3 
Administrative Terminals* 
Ijistallation Schedule 









FY 








76 , 


» 77 


78 


79" 


• 80 

- ^ , 


NATTC Memphis 


• , 3 


4 


6 


6 


6 


NTC San Diego 


1 


3 ■ 


3 


3 


3 


WrC Great Lakes 


2 


3 


4 - 


5 


5 


NTC Orlando , ^ 


0 


1 


1 


1 


1 


mrc Meridian 


0 


0 


0 


•1 


1 


Total 


6 


11 


14 


16 


16 


*Adnujiistrative 


teminals 


proposed 


for the remote sites 


are all 



Itoaeyviell Model No. RCP70 7 Multifunction teiminal, peripheral, message 
switching subsystems equipped vdth disk storage to hold all input/output 
messages, a 300 LPM printer and a 600 CPM card reader. Installed at 
the Memphis site are Honeywell Model No. RCP 701 Terminal/Pteripheral 
systems with specifications identical to those installed at the remote 
activities. 

« - ■ ■ 
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The four models of Level 66 vdll allow the Navy to configure the 
central site ifistallation for the cxrrrent workload and perfonnance 



requirements. The initial installation vdll be configured li^ii^ *^^ 
66/20 processors. By the tijne CMI is fully expanded, it is kiiticipate<l 
that the central site will have to be upgraded to a Ntodel 66/60 two- 
processor system. Detailed equipment lists for the central and remote 
sites are provided in Table 4-1 and Figures 4-1 and 4-2, 

The CWI system was acquired through cwpetitive procurement pro- 
cesses at an estimated cost of $150 fiOO to the Navy. After 18 months 
of efforO^uch included the development of technical specifications 
and workload descriptions, request,^ receipt, and evaluation of vendor 
proposals, live test demonstrations by all responding vendors , four 
vendors submitted the best and^ final of fers . Of the various plans 
offered by these vendors, the plan vtfiich offered the lowest overall 
systems life cost was selected. 

The Navy*s (Ml conputer procurement was a first in many ways.- ' 
The most significant aspect of this procurement is the 95 percent ' 
performance level requirement to be guaranteed by Honeywell for the 
duration of the systems life. 

During the solicitation process, responding vendors were required 
to convert a significant portion of the CMI data base and a^jplication 
'pjxDgrams to their systems and to demonstrate that the proposed system 
was capable of processing the projected CMI workloads. Because of the 
unfeasibility of requiring vendors to install for live test demonstra- 
tion the large number of terminals that would be required in real 
time, vendors were instructed to demonstrate only four "live** 
teiminals. Hie remainder of the workload was also processed during 



the demonstration but vdth the vendor being responsible for siibmitting 
the workload as if it had come from live teiminals. Simulation tech- 
niques used included line connection, queue manipulation, and other , 
bper9,ting system overheads . As part of the acceptance testing and at ^ 
any other time during the six year systems life, vendors agreed to 
rexun the live test demonstrations vdth all live terminals. In the 
event the required processing time or required response times were not 
obtained from the rerun of a live test demonstration, the contractor 
viould pixmde, at no additional cost to the Government viiatever l^rd- 
ware and/or software as required to meet the processing criteria. 

Terms and conditions obtained are consistently better than those 
offered in the GSA contract, e.g., leasts discbunts range up to 35 
percent depending on the type of equipment and time of installation. 
Purchase opticm credits range from 45 to 85 percent. Special, purchase 
(quantity) discounts total over 1.5 million dbllars^ Volume maintenance 
discounts are offered 15) to a 40 percent level. All extra use/charges 
were deleted frbm the contract. Because of these and similar benefits 
obtainable from competitive procurement, ° every consideration should be 
given in future ADPE pTOCuremsnts to going "the long and often dis- 
oourj^ing way". Sole-source, negotiated procurement seldom produces 

similar, if any, discounts, terras, and conditions. 

■ \ ■■ . 

4.3 Current (KI Language 

On the Sigina 9 , eleven interrelated applicational programs shall 
be discussed. Their description as well as their timing gives the 
best concept of the characteristics of the total Ott language package , 
as well as their operational routines on the Sigma 9. The current 
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Navy ,041 systems programming staff has made significant iirproveraent^ 
for efficiencies in the MSU developed programs. Led by Randy Woolley 
and Carl Weaver, the file handling, cluster interactions , and a new 
site line concentrator have been effected. The operative CMI progriemis 

shall be considered. 4 \^ * ) 

1. A-Reg . This |progr|pBn is used to register students for "cq|»rse(s) 
that are include^injme students ' curriculum w Students will 'initially 
be registered into a course from the Administrative terminals . Tfie 
average. onboard load divided by the course length in weeks wili 'deter- 
mine how many students wiil be registered intb the pourse per week. 
Since Oil is an individulalized self-paced instructional system, there 
is a steady flow of students through the system. Thus , the weekly 
input is spread evenly throughout the ^^^eek^ A 1*5 week course with ^ 
an awrage onboard of 555 students will receive an average of 74 
students per day, 

555/1.5 ^370 
37d/5 74 " 

^ 2. B -ASSIGN S This program converts Symbolic course Items into 

'is 

actual course items that are thereby assigned to specific places 
within the course file. ASSIGN jobs normally are treated as batch 
jobs that caA be loaded at the central site and be processed during 
slack periods with vp to 0.5 hour turn around time. Each week of 
instructional material requires approximately 150t course file records 
v^ich require 40 hand-scribed input cards each. Based upon develops 
ment of the current courses , 1/4 of the original input has to be 
modified before./the course is ready for production usage. Therefore, 



7500 cards are required per week of instruction. 

150 X 40 X 1.25 = 7500 
To adueve the projected course inplementation schedule , it is neces- 
sary to pfocess 4.333 weeks of instructional materials per month: 

7500 X 4.333 = 32500 cards 
' 32500 r 11 courses ^ 20.75 » 143 cards 

3. ' c-NPD . This program builds and maintains the course master 
file. The M3D Program is processed in the same manner as the ASSIGN 
Program, the output irm the ASSIGN Program is thoroughly and e:diaus- 
tively edited before bein|^^aced into the course master file. This 
requires many passes throu^ the course file to adequately check the 
bwmching techniques. 

s 4. D-EVAL. Evaluate is the student control program. It provides 
foixr major functions: entry of students into courses, student progress" 
control, inqiiiry into student status, and course time keeping. 

The "enter" function requires one ijijiut record for each student - 
starting the course; therefore it will equal the daily input from the 
registration('proCtess . The "enter" function produces the student's 
initial iewning guide and starts time-keeping functions for the 
student. 

The student's progress through the course is determined by his 
"responses" to assignments. The niinber of student responses per day 
is determined by the course material. Current courses requi^^-from 3 
to '12 responses per day. The expected overall average is 6 student 
responses per day. These responses are coded on an, answer sheet and 
i«ad into a scanning device vMch transmits it to the central xoirputer 
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for processing. An eriDr message or acknowledgment of a properly 
submitted sheet is returhed to the classroom cluster. 

Inputs from the inquiry function were not included in the normal 
workload description but were accounted for by the peak viorkload 
description. ^ 

The course timekeeping function requires 3 cards per day for each 
course. 

The numbex^of interactions required to process 4755 students in 
11 courses is: \ 



4755 students x 6 responses = 28530 
• 117 students entered ^-^^ J17 

Start/Lunch/Stop clocks for ^ 
r the courses 33 ^ 

Z8680 interactions 

Past experience has shown that an average of 14 course ricords 
must, be accessed per response to satisfy the chaified brandling tech- 
niques now :being used. ' ' - 

The length of the student *s new learning guide will depfend upon 
the course mterial and the student's progress vfithin the course. 
Although the lengths of learning guides may vary, the expected average 
is 20 lines. Learning guide length determines the printing device 
either the cluster printer or the administrative terminal. Under this 
concept multiple lesson study guides are routed to the administrative 
terminal, vMle short one -lesson guides are returned to the classroom 
cluster. The initial learning guides^ are always long and are conse- 
qiOTitly requested from and printed on the adbrdnistrative terminal. 

5. E'DEBUG . The instructors use this program to debug their 
course file. It prints every item referenced by the *Tarent*' item. 
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Therefore, it probes eveiy possible chained path that can be taken as 
tl» result of a response to the item. ^The program input volume is 
determined as follows: 

For 143 inputs to the ASGN-M3D process 

114 Adds T 4 cards per add 29 ^ 

29 Changes v 2 cards per change IS 

itOuo 

44 items x 2 paths per it«n = 88 input cards 
Run as required as a debugging program for Programs MDD and ASSIGN. 

6. F'SYST MAST , The system master qjdate requires approximately 
2 input cards per course and is run as required from the central site. 

' 7. G'HIST UPDATE . The student's responses along with infonnation 
about each module completed are kept in a response file, . At the end 
of the training day, the response fiUle is sort^ and merged into a 
response history file. The history file will contain the student's 
responses as long as he is active in the course. This job will 
noimally he initiated at the central site. 
Response File : 

6 responses + 3 module records = ^records per student each day 
History file: 

9 records x student onboard load x course length in days 

8, H -MATRIX , This program gives a student status report and is 
run as required for students in the systan as requested from the 
Administrative Terminal. 

9. I -HIST PRINT > This program gives a formatted print-out of 
the student*? response and module information. It is run for each 

■tt . . - 
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student cosnpleting the course or as required. It can be also used to 
provide the information necessaryl for counseling and^discipXinary 
action. , , 

10. J-K3STER . This program generates rost^ers. It is run daily 
for each student enrolled in the system. Runs nonnally requested from 
the Admiiaistrative Terminals outside of prime training hours. 

11. K"PRH)ICT . This is a monthly regression analysis run against 
data collected about ^e students \A\o have completed each course. It 
is used to predict module learning time and sets the pacing in the 

course. " ^ v t 

.> -^^ 

Q\l Language System ' . . 

As currently o^rkted/ the 11 CMI programs can be considered as 
being quasi-independent, applicational pro-ams. Prom a systems point 
of viWp they are integrated in terms of \vork floK. This work flow , 
as originally conceptualized is presented in Figure 4.3. (Note that 
positive modifications have been incorporated in the interim.) 
Starting in the upper left hand corner, one can observe that the 
omrse development and updite protess takes place. The course Update 
Program edits all Course Development ' inputs , maintains the Cours6 
Master Filep and causes Course Master Sheets, Course Ertor Lists, and 
Symbblic-to-Actual \M,t Lists to be printed. Second, the learning 
guides and student responses interface with the student element 
ultimately f6eding into the student evaluation cornponent, that is, 
the student evaluation component integrates the course development and 
course master file with the learning guides and the student responses. 
The Student Evaluation Program is the main program in the CMI system. 





<^ C Ql 1. 

•4 I) M a> 

<• 5 9 M 

iS52 



•> £ B 0! 






U \J ♦* 

<a 3 1^ 

1/) 1/) 2C 




Hast 
Sort 


\ i 






1 - ^ 







48 



✓ 

It evaluates stixlent responses, captures student response data, and 
causes learning guides ,. errors in student inputs, and special reports 
to be printed. Several student evaluation routines have been 
developed which allow students to respond to multiple -choice tests, 
true/false or yes/no questions , completion questions , and mathematical 
questions which require numerical inputs. These, in turn, iead to an 
ultWte response history which characterizes the student flow. The 
Response History File contains the following information on each 
student, by course, by CMI unit: the day he responded to a given CM 
unit, the orcein which he responded, the amount of credit possible, 
the mount of credit received, the student's estimate of the instruc- 
tional and testing time r«iuired for eadi CM unit, and the entire 
response made by the student. The infomation in the Updated Response 
History is used to generate various reports aiid provides the Wa 
necessary for enqjirical course development and- revision. \ 

In the upper right hand comer there are various activities , 
■ dealing with the student registration process. The complexity of this 
is directly proportional to tJie number of students in the system. The 
Student Registration Program edits the Student Transactions and. causes 
a list of the errors to be printed. Student System Master Sheets are 
also generated by the progrpm and are used for various administrat;ive 
purposes. This program maintains the various student files in the CMI 
system. Finally, this leads to a student history which merges his 
response history with his actual requirements in order to track him 
through the program. The Student History File contains biographical 
data and summary data which includes elapsed training time , beginning 
scores, ending scores , points possible, and i^ints received. As you 
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will find in subsequent sections , this system vdll be re-(^signed and 
iqxiated in terms of the new Navy CMI^ system although these eleven 
essential ftmctions still are retained. 

Language Characteristics 

\ second way to understand the current Navy OH language is to 
code xsp CMI materials. This coding analysis is offered not as a 
definitive description (current modifications will likely make even 
these descriptions obsolete within the very near future) but rather to 
give some concept of how the coding operation is organized within the 
system. A clear understanding of how the coding operation is orgaiuzed 
within the system illustrates its role and gives a clear understanding 
of why the coding operation is a qumbersome and high mai^x)wer demanding 
activity. It should be noted that many of the present requirements 
could be done away with via the use of pre-cojn)ilers , ,that is, a 
program vAiich compiles natural language iiq)ut into the appropriate. 
CMI coding formats or^ via the use of on-line entering and editing (this 
will be available on the Honeywell system) . The concept of on-line 
entering and editing has become a major feature of the new syst«n. 

J 

Foimats For Coding Course Material 

Three record types are used to code infoimatiolh into the CMI 
system. They are designated as Descriptor > Statement, and Response 
Records. Information being coded may require one or more of the 
record types. The coder identifies the information \rfxich he is coding 
by assigning it a six-character code viiich will be used to access the 
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infoxmation in thfe system. All of the informatirti vrfiich is identified 
by the same six-character code constitutes a CMI *'unit'' of information. 

TJie Descriptor Record (there raust be one) describes \A\at type of 
QAl unit is being coded. If the unit requires a student response, the 
type of evaluation to be conducted is specified as well as the action 
to be taken as a result of the evaluation (branch schemes) . The CMI 
igiits which ma/ be accessed are also coded in the Descriptor Record. 
The following paragraphs indicate the types of information to be coded 
in the various card columns. 

Columns 1-3 are used to code a three -character course code. 
For exsattple, the j'\MFU(A) course has been designated as course 001, and 
all CMI units which are prepared for this course will be so designated. 

Columns 4 - 9 are used to identify each CMI unit of information. 
The coders work vdth symbolic numbers vSiich are coded as one alphabetic 
character in column 4 and five numerical characters in coluntis 5*^- 9. 
When units are assembled in the course , they are assigned a subsequent 
number referred to as the ^'actual'* item niinber. The first four 

characters of the ^'actual" item number identify the track on vhich the 

f 

unit is located, and the last two characters identify the position of 
the unit on a* track. ^ 

Column 10 contains a code vtfiich indicates which type of record is 
being coded. Descriptor Records are assigned a code pf '^V\ ' 

Columns 11 - 12 sure used to identify the type of OH unit being 
coded. The first digit indicates the evaluation type, and the second 
digit identified the branch scheme to be used. 

Columns 13 - 15 are used to place the GNU unit in an instructional 
segment (benchmark) . The benchmark locator allows the CMI system to 
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sUBSiarize stxident data when an instructional phase (benchmark) has 
been coofipleted. A Oil unit which is located in a particular benchmark 
cannot reference or be referenced by units vMch are located in 
another benchmark. General conments which are presented frequently 
throughout the course may be Referenced from more than one bendnnaik 
if they do not carry a specific benchmark locator. 

. Colunns 16 - 20 are used with CMI units which expect a response 
' fran the student. Each of the manticipated branch codes is turned on 
with a "1" and off with a "0". When a student receives a score of 
0 percent which i^ans that the student's response to test items did 
not result in partial or v*iole credits being given, the unanticipated 
branch codes are used.^ The unanticipated branch codes are directly 
correlated with the\ Branch Uhit numbers. If the unanticipated branch 
codes are executed and ar^ coded 01010000 , then the information under 
the unit nunbers in the second and fourth positions of the Bran<)h Uhit 
nuiijers field would be accessed. „ . 

Colunns 24 - 29 are used viien the instructional strategy calls 
for an evaluation of overall test results in terras of percent correct , 
and subsequent branching is dependent on the score made. The three 
percent scores allow, as many as four different prescriptions to be 
made. If 63-75-90 are loaded, the four paths available are below 63 
percent, 63-74 percent , :75- 89 percent, and 90-100 percent. Of course, 
different percents may bs loaded, and these will depend on the instruct' 
tional strategy. 

Colunms 30 - 77 are used to code Oil unit numbers vihich may be 
reiEerenced. 
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Coluims 78 - 80 are. used to 4dicate v*iat action is to be taken' 
^regardijig the record. Additioas^to a course file are denoted by a "l" 
coded in cai^ coluinn'78, and columns 79 and 80 are left blank. Changes 
or corrections- to the existing course file are accoinplxshed by the use 
of codes '•2" (insertion). "3" (replacement), and • •4' • (deletion) in . ^ 
columnC8. These actions imost be used in conjunction vdth line numbers 
which are obtained from a document called the .Master Course Listing. 

Multiple Descriptor Records may be used for expanded code require- 
ments. \ / 
Since the CM system is designed t/ iccept short answer, numeric, 
and mltiple-choice test .response^ , the Response Record may be used to 
accommodate any of these. It can also be used vath a variety of 
evaluation and branch schemes. Several, Response Records may be used 
witlTspecific units to code correct answers, alternative correct 
answers , and anticipated W^ng answers , alternative correct answers , 
and anticipated wrong answers. Provisions are also made for unantici- 
pated answers (see columns 16-^23 under Descriptor Record) .The 
following paragraphs indicate the types of infoiroation to be coded ^ 
the various card columns . 

Columns 1 - 3 are used to code a three -character course code. 
Columns 4-9 identify the M unit with which the Response 
Record is asscK:iated. ' ^ 

Column 10 identifies the. type of record. Response Records are 

designated by the number "3". 

Columns 11 - 12 indicate the sequence in which the Response 
Records should be used Oien evaluating student responses. 
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Colunns 13 - 14 are used only viien the student is expected to 
give a numBric, response, and the ins^^ctional strategy calls for a 
tolerance factor to be used in evaluatuig the student response. For^ 
example , if the student were expected to respond vdtii a numerical valud 
to one or a series of problems in electronics and the instructional 
stiEategjr called for a tolerance factor of *5 percent, then 05 would be 
coded in colunns 13 and 14. 

Columns 15 - 17 are used to assign a weigjit to the Response 
Record. The weight is the credit vfcLch a student will receive if he 
receives full credit. If there is only one R^onse Record coded and 
the student misses some of the test items , a percent is calculated and 
multiplied tiines the weight assigned to the response. In other words, 
the student may receive partial credit. 

If several Re^onse Records are coded, the system will compare 
the studmt^s response or i;esponses with each of the Response Records 
(sequentially as indicated by thq sequence minber in colimns 11 - 12) 
and use the Record which gives the greatest credit . Of course , as 
soon as a conq>arisoh results in. 100 percent match, the system uses the 
particular Response Record. 

Coluqfis 18 - 25 are Used to reference other CMI units of informa-^ 
tion vAiose numbers are coded in the Branch unit field of the Descriptor 
Record. Tl» use of anticipated Branch codes depends on the type of 
unit with i^ch the Response Record is associated. 

XGolurans 26 - 27 contain the number of characters vihidx are 
e3q)ecteasin the student response. 

Colunns 28 - 77 are used to code the characters against \ihici\ the 
student response will be evalmted. The mmiber of characters which 




can be evaluated under one CMI unit is fifty at the present time. 

Colunns 78 - ^0 are used to indicate what action is, to be taken 
regarding the Response Record and are the same as described under • - 
Descriptor Record. 

4.4 New CMI Language Functions 

The new CMI langu^e is a conversion of the current one and an 
extension into real time editing and interactive functions; The 
prunary einphasis is on editing and authoring. Extended CMI function 
like siraulati^ shall be developed by Navy personnel. 

The interactive language shall be file c6n5)atible with the CMI 
language and shall implement the editiiig, authoring, and training 
simulation requirements. The language complexity shall be minimized 
so that instructional programmers can learn and perform efficiently. 

Text Editing . The on-line text editor shall support data files, 
FORTRAN, assembler, ai^jDOBOL languages. The text editor will have 
the following Abilities; 

1. To select an existing file or build a new file. 

2. To merge two files or any portion of these files into one. 

3. To edit each record within the file, randomly and allow for 
record change. 

4. To change a record or records including deletion and/or 
insertion of a string. 

5. 3^0 shift a string within a record or records in any direction. 

6. To resequence a file being edited and change the seqtience 
nunfcer and move a specific record. 

7. To update multiple records with an instruction type. 

\ 8. To save an edited file by name. * 
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J. To maintain file sequence numbers in such a way that they are 
transparent viien the file is not in edit mode. 

10. To copy the file creating an exact diqjlicate under an alias. 

11. To furnish accounting data on file (date or last update, 
record size, file length, sequence format, etc.) 

12. To direct output to another designated terminal. 

13. To delete or inserts a ribcord. 

14. To delete the file. ' 

15. To search records craitaining a specific string. 

16. To displ^ any record or records. 

The commitment for interactive problem solving and simulation 
shall focus on CRT display requirements and foimal modeling. 

Conclusions 

1. The current CMI hardware configuration plus language represents 
a minimal cost effective training solution because of the 80\o 1 
student/terminal ratio, the 30 second or less response time, aVa 
basic minimal set of essential CM functions yields full achieven^t- 
of training goals. ^ ' { 

2. The new Navy computer represents a cost effective procurement 
in that a 1.5 million dollar savings plus guaranteed reliability was 
gained vfliile modular expansion \fas insured. ^ 

3. The guarantees on the reliability And capacity of the new CM 
computer are of the highest per^mance value in that Honeywell is 
required to insure 95 pei-cent perfoirnance and guarantee equipment 
si^jport if the benchmaric standards are not maintained. These are 
supported by fiscal penalty clauses. 



ERIC 



56 



4. ".The new m language shall include the features o£ editing, 
authoS^ri6»( ii^teractive training vJiich represents a significant 
increase' in complexity and sophistication. , 
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CHAPTER 5 , Instructor Functions in Navy CM 

With so niudi of the interest in CMI focused on monitoring and 
testing, it is easy to overlook the less salient, but very substantial 
contribution by the learning center si?)ervisor. The fact that learning 
center supervisors (LCS) udo not present instructional material in the 
traditional sense should not be taken to ncan that their input to the 
system is inconsequential vrith respect to learning outcomes. By 
employing the coin)uter to perform a variety of major instructional 
tasks OB.g*> presentation of assignigeritsV assignment of remediation, ^ 
evaluation of perfoxmance) , the LCS, unlike his counterpart in a 
traditional setting, can devote a substantial portion of his time to 
tutoring and counseling students on an individualized basis. Students^ 
appear highly receptive to this type of sv^jport, as evidenced by their 
tendency, in a recent Navy CM course evaluation, to give more positive 
ratings to the LCS th^ they gave to either the course or the course 
materials (Thuimond § Hansen, 1975). Nforeover, these ratings were 
found to be appreciably higher tlaan those received by college professors 
evaluated by their students in convent ional learning environments 
(Freijo § Jaeger, 1974). 

The specific responsibilities of the LCS, although extremely 
numerous and diverse, has been summarized in terms of six 
individual categories: administrative, managerial, supervisory, 
evaluation, diagnostic;, and counseling-. Administrative duties, 
enconpass orienting new students to the instructional system, 
checking daily printouts from the conputer to monitor progress and \ 
assign remediation, and no^fying the Wing Superviso^^vAien students^ 
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have successfully conflated all course 'requirements. The managerial 
and supervisory functions generally consist of helping students to 
adopt suitable pacing schedules and to make proper use of media for 
training purposes. In assuming the role of an evaluator, the LCS 
conducts informal assessments of student performaijces through oral 
questioning, observing their work , and daily referencing of the 



conputer printout. With respect to the diagnostic functions, the LCS 
is e}q)ected to carefully evaluate the materials and methods recommended 
for remedial usage, and select those that appear most appropriate for 
his students. The final category, counseling, involves the very 
in5)ortant responsibility of meeting regularly with students oiyan 
individualize!^ basis in order to identify and discuss areas of conflict, 
to help further their interest and motivation in the course , and to 
assist them in selecting an academic program that is most likely to 
accommodate their career goals and learning capabilities. 

In light of the above, it seems justified to assume that the 
effectiveness of CMI systems will depend to a large extent upon the 
quality (i.e.^ training and experience , attitudes, and overall instruc- 
tional abilities) of its LCS. A systematic analysis of LCS character- 
istics, therefore, must be considered an essential component of any 
serious attempt to describe or evaluate GMT. 

Purpose of the' Study and Data Collection Procedures 

In accordance with the rationale stated above , a primary objective 
of the present study was to gain insight into the personal character- 
istics (i.e., training, attitudes toward CMI, etc.) of LCS's currently 
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employed in the Q4I technical training system at MAS Memphis. The 
basic methodology involved administering a paper and pencil qiies.tion- 
naire to a sample of 123 LCS's vdio presently hold positions in the MAS 
Nbnphis CMI training system. To help insxrre the validity of results , 
precaution was taken to preserve the anonymity of the respondents. 
Specifically, once the questionnaires were administered, the respondents 
were allowed to conqjlete them independently, without intrusion by their 
superiors or by any individuals connected with the present study. Each 
respondent was given a phone number and the name of an individual viio 
could be contacted if questions arose vMle completing the questionnaire. 
Most inqportantly, preliminary instructions indicated that names should 
not be written on the questionnaire form since there would be no 
interest in determining the identities of individuals*. 

The finalized version of the questionnaire contained 31 items , 
grouped accordir^^ three separate categories: (a) a demographic 
section, consisting' of five items, used to collect background informa- 
tion (age, military, and CMI experience, etc.) pertaining to the sample 
being described; (b) an yattitudinal'* section consisting of 15 state- 
ments describing various applications and objectives of CMI (thfe LCS 
reacted to the statements by selecting multiple -choice alternatives 
or, in many instances, by indicating levels' of agreement or disagree- 
ment on a five-point Likert-type scale) ; and (c) a supplementary atti- 
tudinal section containing 11 statements about CMI with the requiranent 
that responses be open-ended. The questionnaire may be viewed in its 
entirety in Appendix B. 
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Results 

For puTiX)ses o£ clarity 'StrI cx)nvenience , the findingSv relating to 
each o£ the three sub-scales conprising the questionnaire will be 
treated separately. 

DeiBOgraphic Data 

The first five itms on tlie questionnaire were used to collect 
basic background information relating to the respondents. In Table 
5-1, the findings show the percentage of sipervisors vAio fall into 
specific categories associated wit;h age, years in the military , military 
status, years e3q)erience with Qtt, and educational background. Since 
these results are mostly self-explanatory, the description to follow 
will be fairly brief in nature. 

The total sairple was fairly homogeneous with respect to background 
characteristics, with age, perhaps, being the only exception. A pro- 
file of a ''typical'' or "representative" respondent would suggest that 
he would be between 21 and 40 years of age^ enlisted in rank, fairly 
experienced in military service but inexperienced in working with 
conputers, and educated through or a few years past the hi^ school 
level. 

Attitudinal Data 

A total of 15 itans comprised this portion of the survey. In 
summarizing results pertaining to items for which the selection of 
more than one response was permissible , frequencies rather than per- 
centages will be reported. 
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TABLE 5-1 
SIMIARY OF DEMOGRAPHIC DATA* 



^14^ : Less than 21 years 20.3^ : 31^5 years 
20| : 21-25 : 36-40 

EM •• 26-30 - 773^ : over 40' 

2. Years in Military 

1-6% : Less than 2 years 22.8% : 6-10"years 
1^ : 2-5 : over 10 

3. Military Status 

95.1% : Enlisted " ■ 
TM : Civil Service 

4 . Experiience with Ccyyuters 

51.2% : Less than 1 year t 

39.8% : 1-3 

" T75T : 4-6 

rr^T : 7-9- . ■ ^ 

5. Educational Status 

: did not finish high sdiool 

56,1% : high^ school diploma 

36,6% : some college experience 

4»9% : college diploma " 

«8% : advanced degree 



* the data are expressed in tenns of percentages of those falling into 
specific descriptive categories , 
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The extent of computer usage in military tra»nin£ . (Item #1,2, 
3, 9) . Four of the items were used to survey opinions concerning the 
extent to,v/}iich computers are, and should be, enployed for military 
training. The 'coi^jlete items statements and their corresponding alter- 
natives can be seen in the appendix. ITie vast majority of responses 
were distributed fairly evenly across the four less extreme categories , 
with the greatest nunfcer (25 .2' percent) reflecting the view that CMI 
presently accounts for 11 - 20 percent of military training. 

More revealing, perhaps , from the standpoint ascertaining attitudes 
toward Qil, were the responses to Item n, indicating the extent to 
vMch LCS's believed that computers should be employed. Approximately 
37 percent of the respondents expressed the opinion that conputers 
should be used much more (24 percent) or slightly more (13 percent) 
than is presently the case ; only 19 percent thought that the amount of 
coanputer usage should be reduced. Tlie supervisors were generally in 
agreenffint in reacting to Item #3, mth almost 85 percent indicating 
that the extent of computer usage will probably increase in the future. 
However, the question of whether the increased use of computers will 
further productivity in training produced little uniformity of response 
almost 40 percent of the sample believed that tlus would be the case , 
18 percent were undecided, and 42 percent disagreed with this notion. 

The overall inpression one obtains from the above data is that 
the typicgiL learning supervisor is more likely to support than disagree 
with the notion that the amount of computer usage should be extended 
beyond its present usage. There is considerable disagreement among 
learning supe^isors on the question of whether computer usage is 



likely to increase productivity in training, however. About haTf feel 
that it will and about 4»al£ feel that it will not. 

Effectiveness of CMI relative to other systens (Items i?4, 6). 
The responses by learning siq^ervisors to Items 4 and 6 indicate an 
extremely positive attitude toward CMI relative' to alternative training 
strategies. Specifically, 61 percent of the sample agreed or stifangly * 
agreed with a statement (Item <?4) to the effect that the feasibility 
and usefulness of CMI are obvious and no longer in question. It is ^ 
also enccRiraging to note that ^en the supervisors were asked to select 
the most desirable instructional mode for military training, GNU emerged 
as the most popular choice (44.7 percent), followed by conventional 
instruction (34 percent) , tutorial CAI (9 percent), and progranined 
manuals (5 percent) . 
► • . 

Problems associate with CMI systems (^Jisnh #5) . The sipervisors 
were asked in Item MS to identify vAxat they perceived to be the two 
most significant problems relating to the present use of compiiter 
technology in military training. Out of the 123 stpervisors tested, 
55 identified ^'instructor attitudes^^ and 54 identified '*cost^* as the 
most important barriers to tlje successful ijiplementation of CM pro- 
grams. The fact that instructor attitudes was the most frequent choice 
is surprising in light of the favorable views expressed toward CMI in 
response to other items (see Itens #4 and #6 p above) . It may be that 
supervisors interpret the problem as one involving poor attitudes by 
their colleagues, even though they, themselves, may react positively 
toward CMI. Other problems relating to CMI were identified as student 
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attitudes (£ = 47), insufficient hardware technology (£ = 37), and 
poor organizational climate = 22). 

i^antages of CMI '(It«n #^2) . The thiree major advantages of CMI 
perceived by tlie LCS's to be (1) savings in instructidnal time • 
(f « 82) (2) flexibility in handling varying training loads (f ° 56) 
and (3) adaptability to individual differences (f = 46). O.ther 
re^^onSaibly popular selections viere that OH: offers greater uniformity 
in the quality of training (f = 39); provides greater assurance that 
instructional objectives vdll be met (f = 29); and saves money (f = 26), 

Characteristics of studmts (I tens #7 , 10 , 13 , 14) .' In responding 
to Item <?7p most learning supervisors (77 percent) expressed the belief 
that CMI is more likely to benefit students viho are average (22 percent) 
or above average (49 percent) in learning ability. Only a small pro- 
portion (14.6 percent) selected the below average student as the one 
most likely to derive benefits from computer-based instruction. The 
learning supervisors, as a whole, also seemed to regard OH as more 
advantageous for enlisted men at .the entry level (f = 68) or beyond it 
(f 5S)' than for cadets (f = 23), officer candidates (f = 26), or 
officers at basic or advanced levels of training (f = 37) . 

The role of the learning supervisor in OH (Items #8, 11, 15). 
The findings to be discussed in this section relate to the supervisors' 
attitudes regarding (a) their own influences in computer-based instruc- 
tion (b> the nature of their responsibilities and (c) the training 
requirements of their profession. With respect to the first of these 

75. ' ■ ' ■ 
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topics (Item #8) the majority of respondents appeared to feel that 
their influence on students and learning outcomes is not as great as 
that of conventional classroom teachers : 31 percent defined their 

r roles as ''considerably less influential'*; 32,5 percent defined it as 
"slightly less influential"; and 19,5 perdent viewed it as conparable 
in influence to conventional instruction. 

Even though the learning si:pervisor may assune a less central 
position in. the classroom relative to his counterpart in the traditional 
setting, his instructional responsiMlities, nevertheless, are numerous 
and diverse. Item #15 on the questionnaire listed some of the more 
iiT?)ortant of these responsibilities and asTced the siqiervisors to select 
the one vtfiich, in their opinion, generally required the most time and 
effort. The results indicated that the greatest number of sipervisors 
identified "answering questions from ktudents" (39 percent) as their 

-rr^sfextiji^^ activity, with relatively feiver selecting (in . 

descending order) counseling stij^ts (2r pSc^ 
dities (19 percent) , providing remedial assistance (12 percent) , and 
disciplining students (about 3 percent) . 

Finally, there appeared to be little agreement among the respon- 
dents as to the best procedures (training requirements) for selecting 

' learning stqiervisors for assignment (Item #15) . Many thought that 
personality and attitude test scores conprise extremely useilul criteria 
(f « 41) , but almost an equal number (f = 37) indicated that supervisors 
should be selected for assignment sinply because they volunteer. Less 
si^jport was voiced fbr selection criteria based on NDS or skill i4enti- 
ties (f « 27), prior teaching experience (f ° 24), availability of 
candidates (f ° 11) , or prior knowledge of ccHnputers (f =8) , 
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Sumnary . In general, the attitudinal dal^ suggest that most of 
—the learning supeivisors tested react favorably to computer-based 
^^instruct-i^n, Viewing it as- preferable to conventional ins trulst^on and 
believing that its usage should (and will) be extended in the future. 
The consensus of opinion appears to be tliat CMI is mast, applicable 
(beneficial for) to enlisted -personnel, but is ^Sely, by itself, ^ 
to comprise a cause for increased enlistments by eligible candidates. 
The major' advantage of CMI was identified as savings in instructional 
^time, i^ereas the major disadvantages were viewed as instructor atti- 
tudes and cost. The supen^isors, for the most part, view their role 
as less influential than that of the conventional ijistructor and feel 
that it primarily involves , in terms of time and effort , answering 
questions from students, counseling, and basic administrative duties. 
Selecting individuals for assignmejit as supervisors should be base<^ 
according to most respondents , on personality and attitude test scores 
and according to whether or not the candidate volunteers. . 

V 

. Supplementa r y Attitudinal Data (Open-ended Responses) 

the open-ended~secrtoR~o£^-ti:^:5uestionnair^ encrarpassed eleven 
questions, all requiring responses tliat could general^hLjbe phrased 
using one or two key words or, at nrast, a brief sentence. Tliese 
responses were evaluated by two independent judges and grouped sepa- 
rat.ely for each question according to similarities in content (or 
point of view). From these groupings, the percentalges of LCS's expressing 
specific views could be detemined. In the interests of brevity, and 

"recognizing that there was a fair amount of overlap in what was measured 
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in the two attitudinai sections o£ the questionnaire , the present ^ 
summarization of results vdll be mostly restricted to a reporting p£ 
the actual numerical data with little attempt at interpretation or 
' verbal description: 

Question #1 : ,How important is the role ^£ the learning siper- 
visor? General reaction was that sipervisors' role^s; inportant: 



Responses: 

Very Uhiraportant \l3.0% 

Uhimpprtant 3.3% 

Neutral 6.5% 

Important 2.4% 

Very important* 47.2% 



^Percentages may not add to 100 percent due to the fact that 
some respondents left items blank. / 

Question #2 : How helpful was on-the-job training in preparing 
you for CMI? Opinions were divided, but more respondents seencd to 
feel that on-the-job training was helpful rather than not helpful in 
preparing for CMI: 

Responses : 

Not helpful 20.4% 
No opinion 15.4% 
Helpful 34.2% 

Question #3 : How useful was Instructor Training School in 
preparing you for CM functions in the areas of instructional tech- 
niques, materials, testing, and ISD? 

Most respondents regarded Instructor Training School as not 
particularly helpful in the above areas. The data shown below relate 



specifically to the area of instructional techniques , but are generally 
representative of the results obtained for the three remaining areas 
— materiails , testings and ISD: • ^ , 



Responses : 

Not helpful • « 35.01 

No opinion 9.8% 

Helpful 10.61 



Question J4 : What proportion of students seem unconcerned with 

trying to succeed? 

The general reaction was that less than 20 percent of the students 

did not try to succeed: 
Responses: 

Less than 20% 48% 

Between 20% and 50% 9% ^ 

More than half 5.2% 

Question #5 : What techniques do you use to motivate students? 
\. The TTtost popular responses were: 

1, Counseling ^ "^^^ ^ 

2, Threats or fear (f. ^5) 

3, Appeals to. self -pride (f = 7) 

Question #6 : Overall, how successful has implementation been 

in your training program? 

The general reaction was that inplementation has been reasonably 

successful: . 
Responses: « - ' 
Unsuccessful 11.4% • 



No opinion ^ 23;6% ^ 

Successful ^ ^ j 32.5% 

Question #7 : Has th^"^ system increased the productivity of 
the training program? 

Most of the learning supervisors felt that productivity had been 
increased: 7 

Responses: 

No " 1^ 9.8% 

Sometimes ^16.3% ^ 
Yes 47.2% • 

^Questiojii #8 : How successful is Oil in achieving instructional 
objectives? 

Most responses suggested that the sxiccess of CMI in achieving 
instructional objectives ranged from "fair" to "good" : 
Responses: y 
Poor 12.0% \ ^ ^ 

- Fair 25.0% 

Good 30.1% , 

Question #9 : What are the most successful features of On? 
The most frequent reactions defined the most successful features 



as: 



(1) time saving (f = 27) 

(2) self pacing * (f = 23) . 

(3) reduction in paper work (f = 10) 

Question #10 : What Cre the least successful features of CM? 

The most frequent responses were : 

(1) united student/instmctor relationship (f » 21) 
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C2) unavailability of conputer (£ 14) \ 
(3) low retention o£ material (£ = 13) ^ 
Question #11 : Additional cominents? 

Aim>ng the more popular responses were j that there is. some need' for: 

(1) more on-the-job training (f = 16)1 v 

(2) an alternative system to Oil (f_ = 9) 

(3) ways to increase student irotivation (f = 9) 

Between Groups Conpariso ns 

Additional analyses of the questionnaire data were performed to 
test the hypothesis that LCS's might react differently toftrard CMI as a 
functicSn of (1) the particular school to v^ich they were assigned 
(BE/E; AFUN; ADJ) and/or (2) their chronological age (Ijess than 25 
years; 25-35 years; over 35 years). The data wie re analyzed for each 
item separately via a simple one-way analysis of variance/ 

The findings (the outcomes of 38 analyses) offer little support 

'J 

for the above hypothesis. Conparisons between the three schools 
yielded significant differences (p^ <4)1) in responses for only three ' 
of the 19 ,i terns. The direction of the data suggested that ^LCS's 
representing the' BE/E school were more likely than their counterparts 
to feel that the implementation and usage of CMI should be extended 
(Pt. II: Items #2,4); and that QH has teen successful in achieving 
instructional objectives in their training system (Pt. Ill : ftem # 8). 
This is understandable in that BE/E has ha4 the'^longest experience 
with the current version of CMI. The reaider should be warned, however, 
that these results must be viewed with caution since they were not 



corroborated by outcjomes for sitoilar items and because the very large 
niinber o£ independeiit analyses conducted prevented the control of the. 
familywise risk of^a Type I error. As far as conparisons between age 
sxoups were concerned, the findings, were even less conclusive: - no 
significant differences were obtained in mf^of the analyses. On the 

/ , , / ^m? . .. 

basis of these results , it seems reascMiable to conclude that neither 
type of school nor chrOTK)log;ical age were significant determinants of 
how LqS personnel tended to react toward their profession and toward 
CMI in general. 

Conclusions . • 

1. The learning center si?)ervisors have a high positive >orienta- 

' ' ' t- * . 

tion towards CMI and have clearly committed themselves to its' pro- 
fessionally demanding role . 

2. Students rate the learning center supervisors and CMI iA 
highly positive teims; in fact, these ratings are more positive than 
CQnunoonly found in collegiate atmospheres. 

3. The learning center , supervisors perceived the bisnefits of 
tte'iM system in terms of its objectives (e.g., tiine saving) or their 
role (tutoring and counseling) rather than in the assumed functions 

of test scoring or registration. 
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. CHAPTER 6 : Instructional Systems Development for Navy Qfl / 
Critical to the iirqDlen^ntation of the Navy technical training CWI 
"system is the conversion of existing convention;^l courses into a 
conputer-based individualized mode. This conversion requires personnel 
capable of performing the various functions relating to the Instruc- 
tional Systems Development (ISD) process with the added capability of 
coi^juter coding and debugging. Similar to the investigation of the 
Learning Center Supervisors p this chapter focuses on the personnel 
implementing this course conversion and maintenance process. 

"The Instructional Systems Development (ISD) personnel phase of^ 
the Nayy M utilized a survey strategy, A questionnaire was developed, 
and administered to a sample of ISD persOTnel to solicit information 
relative to the OMI operations at NAS Menphis, The Instructional 
Systems Development Personnel Questionnaire was developed by the 



project team and reviewed by training personnel at NATTC for refine- 
ments. The questionnaire consisted of 16 items that solicited open- 
ended responses. questions focused on areas such as the roles and 
tasks of ISD personnel, strengths and weaknesses of the developmental 
procedures used, accomplishments of Q4I, and hindrances to CMi; ^ 



. The questionnaire was distributed to a sanple of ISD personnel at 
NATTC. Thirteen respondents completed the questiOTniire and returned* 
it. The responses were synthesized for each questiorl for presentation 
in this report. The results are presented in suranaiy form in the 
following paragraphs . . ' 
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QUESTION 1 

How well does the ISD Model serve in the implenffintation o£ CMI? 
The responses are summari'^ed as follows: 

Very well' 6 (46.2%): 

Moderately well 5^ (38.5%) 

Not familiar with ISD Model 2 (15.4%) 
About 85 percenl^ot the respondents- indicated that the ISD Model 
serves as an adequate guide for in^jlementing CMI. The other respondents 
were not familiar with the ISD Model. Typical responses were 'Very 
effective" and "adequately as a. broad guide model."" 
QUESTION 2 ' 

What are the major problems encountered in preparing the curricular 
materials for CMI? 

The responses were grouped apd tabulated as follows : ^ 



Setting job tasks __J'^ 5 (38.5%) 

Setting terminal objectives 4 (30.8%) 

' Preparing^ appropriate materials ^ 3 (23.1%) 
Determining content to be covered 2 (1^.4%) 
Obtaining capable writers ' 2 (15.4%) ^ 

Overcoming resistance to change 1 (7.7$ ) 

(The percentages of "responses add to more than 100 b^ecause some^ respond- 
ents identified ^re than one problem.) 

V. . - ...» , , ■ 

"The principal problems encountered in preparing CMI curricular mater- 
ials relate to providing tasjc analyses of job perfoiniance 'and establishing 
temiLnal objectives for courses. Less frequently mentioned Rfoblems - 
concern preparing appropriate materials, deteminiijg the content for , 
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materials, and competeixcy o£ writers. Examples of cx)manfients are: 
^•proper job tasks," "determining teminal objectives," "determining 
\A\at material is required to attain objectives," and "trying to deter- 
mine what is to Ije covered." 
QUESTICTj^ 

* • «> 

What aspects of the curriculum are adapted to CMI more easily? 

The compilatioih of 19^responses to the question were summarized 

as follows : 

Knowledge 7 (53.8%) 

Theory 5 (38.5%) HP" 

Classroom instruction . 3 (23.1%)^ 

^Testing 3 (23.1%) 

All written jnaterials 1 ( 7.7%) 

Knowledge and theory are dearly the areas o£ learning that are 



seen as most amenab-le to CMI.' Classroom instruction, a more gengral 



area, and testing are also seen as appropriate for CMI. Typical 
responses are: ^ ''knowledge behavior," "knowledge and theory instruc- 
tional areas," "classroOTi th^ry type lessons," "classroom situations 
and "evaluation."' 

QUESTION 4 . ^ . * 
"What are the most difficult tasks in inplementing a QH system?' 
The results of the tabulation of responses to\ this question were 
as follows: ^ • 

Teaching manipulative skills \ 5 (38.5%) 

^ — Overcdming^esistance of staff x ' - . 3 (23,1%) 

- Teaching on-the-job skills ' ^ ]2 (15.4%) 
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^ Achieving uniform quality of materials 2 (15.4*) 

Training instructional personnel 2J[15.4%) 
Pijpcuring facilities, hardware, and materials. 2 (15.4%) 
Ckxnplying to administrative requirements 1 ( 7,7%) 

The respondents perceived jtKat the greatest difficulty vras encoun- 
tered in teaching certain skills: manipulative and on-the-job. A 
relatively small percentage of the respondents also mentioned difficulty 
in achieving uniform quality of materials, acquiring resources, and 
training instructional personnel for CMI operations. Illustrative 
coranents are as follows: "teaching manipulative skills," "overcoming 
resistance QQ by conventional instructors and managers ," 'Job 
skills," "training the required personnel," and "creating progranmed 
instruction to a uniform standard." 
QUEgnON 5 



How could^ the implementation of a CMI syst«n be inproved? 

The cornnents of the respondents are suranarized as follows: 

More cooper-ative efforts among 5 (38.5%) 

personnel at various levels 

Provide specialized inplemenjtation 3 (2L.1%) 
per^Wiel 

More training of involved personnel 2 (15.4%) 
More assets 1 ( 7.7%) 

Create settiiig for tryouts 1 ( 7.7%) 

V ..jv i^fjo conment 2 (15 « 4%) 

The three most freqtfcfttly mentioned ways of inproving CM' inple- 
rawitation involve personnel. Mbst frequently cited was the need for 
more cooperative efforts among the persoraiel involved. Provision of 
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specialized implementation personnel was mentioned by about one- fourth 
of the respondents. Examples of the comments were "getting all concerned 
to work together," '^by coordination and understanding of the system by^ - 
all parties concerned," "have a cadre of skilled people do the whole 
job or closely supervise it," and|^*more training of school adminis- 



trators and personnel tasked with implen^ntation." 

QUESTI0N_6 , ' 

How does CM improve the quality of instruction? 

The tabulation of responses produced the following results : 
Standardization of course content 8 (61.5%) 
Individualization, self-pacing 4 (30,8%) V ^ 

Efficiency of time and effort 3 (23.1%) 

Unbiased ' . 1 ( 7.7%) ^ 

The principal inprovement in the quiality of instruction by using 



Qll was identified as the uniformity of content that is taught in 
individual courses. The capability to^ndividioalize /instruction and 
the efficient use of time and effort were also menti^ied ways that 
CMI contributes to instructional improvement. Typical comments by the 
respondents were "teaches ea|^ student the same material," "self-paced 
and individualized instruction," and "saves time for the student and 
the instructor." ^ 
QUESTION 7 

Miat is your reaction to the procedures utilized in iirplementing 
CMI in your training program? ' ' 

The summarization of the^ respondents ' comments produced the 
following results : > . 
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11 

Positive . 10 (76.9%) 

Negative ^ 1 ( 7.7%) ' 

^, ' Ndr oonanent 2 (15.4%) . 

Three -foiarths o£ the respondents made COTnnents of a general or 
specific. jiature thaVwere positive. Only one oonnent was negatiw in 
tonC Scffle^^examples of positive comments are ''the procedures were 
well-plann^ and properly utilized," excellent when trained personnel 
are used," and "too much eiqphasis on empirical development and not 
oioug^ an ^sessment of feedback." 

QUESTK^ 8 

Ifliat roles do you fill in developing CMI materials? 

TTie responses td this question revealed the following roles: 
Task analysis 3 (23.1%) 

Writing materials 3 (23 .1%) 



Coordinating instructional procedures - ^ 
and software development 2 (15.4%) 

Course coding . 2 (15.4%) 

Inplementing ISD 2 (15.4%) 

Editing materials " 1 ( 7.7%) 

Siqjervisory programmers 1 ( 7.7%) 

Test development , 1 ( 7.7%)- 

No conment 1 ( 7.7%) ^ • 

The cooinents^of the respondents reflected that a great variety of 

roles are filled in developing CKE materials. Task analysis and writing 

course material's were specified by about one-fourth of the respondents. 

Less frequently mentioned were functions relating to coordination of 

instructional and software developannent , course coding, and editing 

materials . 
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CATION 9 

How many man-hours are r^^uired on the average for you to prepare 
one hour of CM materials? ' ^ 

The answers of respondents were as follows : 



10-12 


1 ( 7.7%) 


30 


1 ( 7.7%) 




1 ( 7.7%) 


70 


1 ( 7.7%) 


150 


1 ( 7.7%) 


Unknown 


,. 4 (30.8%) 


No coDiinent 


4 (30.8%) 



. The responses concerning man-hours required for producing CMI 
materials were highly variable, ranging frQm 10 - 12 to 150, The responses 
were exemplified by the following: ''40 hours average, depends on the 
nun4>er of objectives for the particular lesson;" "unknown, depends on 
material being prepared/' and "from start to finished product, one 
would estiinate 30 hours." ^ / . 

QUESTION 10 ^ 
" What tasks do you perform in developing CMI materials? 

The compilation of responses provided the following results: 



Develop or write materials 

Analysis 

Edit materials 

Code materials 

Identify and develop tests 

Validate materials 



5 (38.5%) 

4 (30*. 8%) 

2 (15,4%) 

2 (15.4%) 

2 (15.4%), 

2 (15.4%) 
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Design materials " 1 ( 7.7%) 

Review materials 1 ( 7*7%) 

No cxOTwnt 3 (23.1%) 

The tasks that were identified by the respondwts were varied, in^ 
nature. About fwo-£i£ths of the respoi^ents indicated that they develop 
or write instructional materials for CMI vMle about one-third referred 
to analysis of the learning tasks to be acconplished. Other tasks 
included functions siSch as editing materials, coding materials, test 
development, and validation. 

The responses revealed three primary areas of difficulty in 
developing CMI materials: developing objectives from task analysis 
results, utilizing clearly understood language in materials , and 
assent lying materials and aids needed for the courses. Illustrative 
comments were "conversion of each task selected for training into an 
instructional objective," "presenting the material in language easily 
understood," and "getting all materials together that are to be covered. 
QUESTION 11 • . 

What are the greatest difficulties encountered in developing CMI 

materid^s? ^ ^ 

The comments relating to difficulties vtere siaranarirfed as follows: 
Converting tasks to clear objectives 4 (30.8%) 

Using understandable language £ 

in materials \ ^ 4 (30.8%) 

Assewb lying. miterials/aids 4 (30.8%) 

Starting development precess 1 ( 7.7%) 

/ Validating materials / ^ ^ 

Resist^e to diange 1 ( 7.7%) 

• - <^ 
No otranent 2 (15. 
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■ QUESTION 12 • ' " 

What are your most rewarding activities in developing CMI .. 
materials? 

The comnients of the respondents were tabulal^d ^d.th the following 

synthesis resulting: 

Effective instruction 9 (84.6%) 

Developing instructional media 

and materials ' 2 (15.4%) 

No comment - 2 (15.4%) 

fvtost of the respondents suggested that the accomplishment of «> 

effective instruction is tKe most satisfying aspect of developing pn 

materials. The~materials and media of instruction were the reward for 

a relatively small proportion of the respondents, - Typical cora]T©nts 

arej 'seeing individuals progress," ''a good final product," "seeing the 

mterials* working in tlie learning centers," and "development of 

instructional media." V 

■ ijUgSTION 15 

How successful is the CMI system in achieving instructional 

objectives? 

^ The categorization of responses to the question produced the 
fallowing results : . ' ' . 

- - Very successful * - 9 (84.6%) 

.Moderately successful ; 2 (15.4%) ^ . 

No comn^nt " ^ . ^2 (15.4%) 



The respondents e;^ressed^very favorable views regarding the 
sucfcess of Q1I in reaching the instfuctional objertives tliat it 
addresses, Exan5)les of- tlieir comments are "highly successful p" "very. 
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good for basic information "success is in proportion to criteria of 
testing," and "very successful in theory- type lessons." t 
9UESTION 14 

What types of resistance to CM do you encounter in your training 
program? 

The coranents of respondents regarding resistance vrere summarized 
as follows: 

Opposition to changp by 9 (69.2%) 

personnel involved ^ , . ^ 

Opposition to CWI philosophy/^ 4 (30.8%) 

systeaoas approach ^ 

Uhspecific 2 (15.4%> 

No comment 1 ( ^•7%) 

The principal source of resistance to CMI appears to be based on 
opposition to change from personnel in various roles and at various 
levels. Some resistance also is derived from opposition to the . 
philosophy of CM and the. systems approach to instruction. Illustra- 
tive coniBents are that "ti^nendous human inertia/' "people's resistance 
to change from platform situations to CMI/' "extreme resistance from 
^ seme persons involved vdth part curriculum and lockstep teaching," and 
* "personnel who do not believe in the sys tans approach." 
QUESTION 15 

Overall , how much job satisfaction do ^you find in developing CM 

systems? ' ' 

The responses to this qiiestion visre classified and tabi4sit©d as 

folloJJS : 

Abw© average' / • 9 (69.2%) 

Average / ' 1 ( 7.7%) 

' . ■ ' /' ' ■ ' ■ ■ ' * . f * 

No ocasnt / ^ * ' ' 3 (23.1%) 
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More than two/ thirds o£ the respondents indicated that they . 
derived above average job satisfaction from their work in developing 
(Ml systems. Typical comments were "personally-- a lot/* 'Very satis- 
fied with my role in developing instructional materials "the most 
satisfying and challenging endeavor encountered in technical training,'' 
find it gratifying/' and ''a sense. "bf accomplishment by producing 

well -trained Navy men/' - 

■ . ( • 

qUESTIQN 16 ■' 

Additional comments about OH. - A 

About one-half of the respondents made additional cop con- 
cerning on. The comments were summarized as' ^llo^ ^ 
"Positive ,1" - 4 (30.8%) * " 

V " Need more suppcnrt . 2 (15.41) 

No comment \ 1 (54.8%) 

/ One-third of the ccmiments made were positive ift tone vMle others 

related to support for,Q1I, The icomments are exenplified by the 

\ \ 

following: "It's the going thing "the greatest aid to mass media 

\ 

since the comic book," ''the CMI metSiod of teaching is the method of 
the future," and "more formal training is needed in Oil concepts and 
skills for administrators of schools and courses." 0 
CONCLUSIONS 

1. CMI is perceived as the best Navy alternative for individualizing 
the training process ; therefore , curriculum specialists like Learning 
Center Si^rvisors rate it highly and are corranitted to its implementation. 

2. While there are personnel resistance factors, the majority of 
the curriculum specialists rate their role^^ as challenging, within the 



ISD mission and professionally rewarding. This highly positive level 
is rather unusual for conputer based systems (see Chapter 8) and reflects 
the high proportion of uniform personnel siqjporting the effort. 
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QIAPTER 1 . Cost Benefits to the Navy from the CMI^ Sy stem 

Cost benefit analysis has many forms ,^ each of which reflects the 
types of questions being addressed. For our purposes , this section 
shall conpare conventional instruction (CI) vdth CMI. It is precisely 
the outcoTOS of this comparison that lias caused the Navy to npve^ 
aggressively forward into aJ3slI operation. To perform this GI-XMI 
coirparison, the projected cost benefits as proposed in the original" 
justification to the CNO shall be considered. In turn, the unfolding 
tost benefits as reflected by the FY IS training data coming from NAS 
Mefiphis shall yield our current level of cost-benefit aggregation. 
Finally, the benefits to the operational Navy shall be discussed. 

Projected CM System Cost Benefits ^ . 

This sec^tion shall define cost elements so that the cost benefits 
can be derived. Within NATTC Memphis thete are three cost elements , 
exclusive of student salaries, which contribute most heavily to the 
training cost per student. These 'elements are instructors, direct 
support p and indirect sipport pJersonnel^ salaries .. (Minor cost areas 
are supplies, travel, etc.) The instructor and direct support costs 
are those directly identifiable with a given course. Indirect support 
includes all other NATTU Manphis personnel. * ■ 

' The number of instructors required for a given course is a function 
of three variables : Instructional hours required at given student/ 
instructor ratio; annual training requiren^nt in number of students ?' 
and class convening frequency. 

Support requirements are, in the aggregate, a function of the ^ 
number of ^students on -board. The planned students on-board to direct 
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support ratio, at NATTC Moiphis in 1972 -was 24.2:1. The planned student 
input to indirect support ratio at NATTC Menphis in 1972 was 12.43:1. 

Courses to be converted to CMI were so ^osen because of their 
ijnpact on the Technical Training Conmand in terras of resources required. 
The FY 75 training plan calls for an average student on- board (AOB) 
load of 20,261 for these courses. This requires 2,691 instructors and 
2,467 support personnel at a cost of about $29,043^,961 for insti^tors , 
and $30 ,615 ,181 for st?>port personnel per year. Student salaries are'' 
about $121,566,000. This totals to about $181,l257l44 per year for 
student, instructor, and support personnel salaries and benefits. 
These costs do not reflect local conmand, facility, travel /CNTECHTRA, 
and CNET overhead. ^ t 

The courses put under CMI demonstrated significant .course length 
reductions conpared to conventional instruction. (See Chapter 3, a 
46,9 percent reduction.) A conservative 20 percent reduction was used 

for this FY 74 analysis rather thap the targe tted 30. percent set as an 

t' . ■ ■ ■ 

objective. Course length reduction has a direct proportional effect 
on student average on-board (AOB) , which is used to derive instructor ^ 
and support requirements. Consequently, it becomes apparent that even 
a very modest course length reduction of 20 percent has an iirpressive^ 
gross cost savings/avoidance potential on the order of several million 
dollars per year. CMI reductions in instructor requirements^ for a 
projected 20 percent- course length reduction are generally on the 
order of 20 percent. This is due primarily to Idwer ApB^s realized 
as a direct function of course length reductions and, secondarily, to 
greater instructional efficiencies. 
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Curriculum developnent constraints (personnel, funds , facilities) 
coupled witli liardware acquisition projections dictate' an inplenientar 
tion plan spanning a five year period. The projected inplementation 
plan is presented in Table 7-1. ^Each projected course conversion by 
fiscal year is .listed. (Obvious exceptions h^ve occurred, for example, 
,<the ADJ course, is now operational although it was not scheduled until 
FY 77,) . ' . ' 

*Cost Objectives- ^ . 

i^n has two primary cost objectives in addition to individualized, 
-training : course length reduction and an increase in student to 
instructor ratios. The objectives are both manageable and' can be 
quantified. w v 

Course length reduction provides th^ most significant gains and 
is directly measurable . Each course to be implemented was or is 
presently being taught under conventional instruction. The number 
of hours re^uiired to teach each course is well documented.. Under Qtl , 
after a statistically significant number of student^s have completed 
the course,, an average conpletipn time may be computed. The difference 

between the time r^uir^d^^ coirplete a giyen course of insttuction ^ , 

• \ / ^ ^ ' 

under conventional instruction (CI) and thei average ^completion time 

\ -• ■ 

mder CMI is the dours^ length reduction. A consep/^ative course length 
^reduction of 2Cf percantNoverall shall be used for analyses,' a reduc- 
tion of 30 percent is the primary objective of CMI. 

• For each course under consideration the CI student to instructor 
i^tio used for theory /practice instruction is 25:l.>JMs is both the*, 
classroom standard oK^gu^^ and tlie ratio normally used to determine 
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Table 7-1 



Projected CMI Course Loadings 



Course Length 
(# of 



Estimated / 
Students under Instruction 



FY ^ 


School 


'Location 


weeks) est. 


^ CI 


CMI 


t TOTAL 




» 






20667 


\ J. 


20667 


74 


BE/B ' 


Memphis- 


5.4 ^ 


19876, 


\ 633 


20509 


74 


AFUN 


Men^ihis 


J 0.9 ^ 


190bl 


■ 1333 


20334 


74 


'SPLICE 


San Diego 


r 5.4 


18028 


2111 


2Q139 ' 


75 


AV(A) 


Menphis 


3.0 


17650 


2413 


20063 


75 


SPLICE 


Great Lakes 


' - 5.4 


^16171 


2596 


19767 


76 


AV(A) 


Menphis 


12.8 


15439 


4182 


19621 


7t 


vSPLICE 


Orlando 


18.0 


15106 


4449 


19555 


76 


SPICE 


San Diego 


18.0 


14016 


5306 


19322 


76 


SPLICE 


Great Lakes 


» 18.0 ■ 


12016 ' 


6903 


18919 ■ 


76 


Nft/BT 


Great Lakes 


' 9.0 


9572 


^853 


18430 


77 


AM 


Menphis 




894'5 


9360 


18305 


77 


AE 


Nterphis 


' 12.0 


8332 


9851 


. 18183 


77 


AD 


Jfenphis 


• 5.6 


7,959' ^ 
7231 


10150 


18109 


77 ^ 


HT 


^San Diego 


19.4 


10733 


17964 


77 


RM 


San Diego 


>1.2 


^ 6244 ' 


11522 


17766 


77 . 


EM 


San Diego 


> /9.6 ^ 


5640 


12006 


17646 


77 


08 . 


' — A T 1 — ^ 

Great Lakes 


712. 0 


4973 


12539 


17512 , 


77 


FT 


Great Lakes 


/ 21.6 


4350 


13038 


17^88 


78 


AFTA 


' Menphis 


2d. 8 


, 3693 


13564 


17257 


78 


AC 


Menphis 


10.4 ~ 


3502 


13717 


17219 


78 


AD 


Menphis 


8.8 


3215 ' 


13946 


17161 


78 ' 


AW 


Menphis - 


'9.6 ^ 


2975 


14138 


1711=3 


78 


IC 


Sqp Diego 


10.4 


2680 


14374 


17054 


78 


CS 


San Diego 


6.4 


2409 


14590 


,16999 


78 , ' 


EESIMP> 


Great Lakes 


17.6 


1203 


15555- 


16758 


79 


CLERICAL 


Meridian 


6.8 


405 


16193 


16598 


79 


GM ' 


Great Lak^s 


9.6 


0 


^6517 


16517" 



BE/E ^ Basic Electronics/El^qtricity - 
AFUN Aviation FundamentUs - 

3^LICE Systems Planned Learning Individualized Core Elements 
AVfA) Avionics 

M4/BT ' Mechanical Mate/Boiler Technician 
AM ^ Avionics Structural Mechanic 
AE Avionics Electrician* s Mate 

AD Avionics ^(bchanical Mate « 

HT Hull Technician 

RM Radio Mate - " 

m Electrician Mate 

OS Operation Specialist 

FT Fire Controller Technician 

AFTA Advanced 1st Term Avionics 
AC Air Controller 

AD ^/Avionics Ordinance 

AW ^vionics ASPA 



IC Internal CcOTfjaxnications- Electrician 
CS ' Commissary '^Specialist 
ETSIMP ^Electronics Technician Sinplified 
GM Giamery Mate < 
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instructor billet requireijaents. Under (Ml the objective is to raise 

this ratio to 30:1. Again ;Jthis objective is directly measurable by 

observing the" number of instructors ^required with- CI and 

• • • 

Cost Elen^nts 

Costs are presented telow for the estimated eight year life cycle 
'of the system for the accepted alternative: ' ^ • 

(1) Continue with Conventional Instruction 

(2) CM! with lease -purchase ADPE 

To determine the taxpayer cost to train a student there *are liine 

i ' ■ ' . ■ ^ ' . ; 

cost elements! to be considered. These are presented in Table 7-2. 



Comparison of Projected- Cost for CMI and CI By the FY 74 Plan 

For the purposes of this projected analysis, only the cost savings / 

in instructor an^ support personnel pLus that attributable to avoidance 

in student salary given the reduced student load were considered. . ' 

Table 7.3 presents the CMI reduction of instructor and support billets. 

(It should Be noted that after six years the instructor and si^jport 

reductions stabilize.) As one can see, the total savings for this six 

year peric^ is $13 million plus and the annual savings range from 

approximately $1 million through $4.4 million. 

In tunip^ the student billet reduction is indicated by the reduc- 

> '. - - 0, ' , 

tion of the average pn-board load, drops from 20 ,667 to 16 ^17. ^ This^ 

equates to ^ avoidance of approximately $22 million per year or $148.6 
million for the 8-year period from FY 74 through FY 82.^ / 

Therefore, in calculating the * total costs savings, pne c^n per- 
ceive that the cost for conputer managed .instruction, will be $426 ,603 /)00 
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v^Table 7-2 / 
Cost Elements for QH and CI 



(Ml ElenenJ^s 


— 1 

Gannon Elements 


^ Li Clements 


ADP 


Supplies ^ 


Instructors 


^ Less Instructors 


Travel < ' A 


Stqjport 


Less Sippgrt 


Plant 


> 


Course Convprsion 


Indirect overiiead 






QJET/CmT overhead 
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Table 7.3 

Projected Cost Savings -for Instructors 
and Support Personnel in C^I 



FY 



Instructor Support Billets Total 
Bill&ts Mil ' Civ Billets 



Itollar Cost 
Savings . 
MP, N 05MN 



Annual $ 
Savings 



75. _ 


87 


59 


8 


-154 


1652 


119 


1771 - 


76 


155 


102 


13 


270 


2891 


208 , 


3099 


77 


221 


149^ 


21 


391 


4163 


300t 


4463 


78 


131 


88 


11 


230 


2484 \ 


179 


2663 


79 




35 


^ 5 


^4 


1011 


72 


1083 


80 


A 




JL 


10 


LOO ' 

f — 


_I 


107 


Total 


654 


436 


59 


. 1149 


12301 , 


885 


$13186 



\4hile the cost for conventional instruction is $625,870^000 , a 
difference of $199 ,267,000. Thus, the initial j^rojectecl analysis 
indicates that the CMI alteraative provides significant cost savings. 
It should be recalled that the projected ^cost savings are^ fi^recy at 
the 20 percent level to be conservative , but in fact , the savings are v 
operating at 46.8 percent currently. Turning from these projected 
cost benefits , the realities of FY 75 yield cost benefit figures of an 
even more positive character. , " ■ * 



CosX Benefits Achieved For FY 75 

As presented in the pripr section, the cost benefits for the Navy 
CMI systein ld6k most advantageous i As with any system, though, the* 
reality of its, beginning implementation determines whether the projected 
'benefits are being realized. This section speaks directly -to this 
re^ality. • ' ^ _ ^ 

For comparative purposes , the fisCai year 1973, just prior to 

beginning operational implCTientation of CMI shall be used. Therefore-, 

the foil oidjig analysis fgr FY 73 as opposed to FY 75 is composed 

within the four CMI objectives. ^ ' 

Reduced Course Ti me. As presented in Table 7.4, one' can see that 
^ , w 

the number of AOB students has increased, and given our awareness that 
for each of the courses there has been a significant drop in length. (a 
reduction of 46.81), this yields a savings in student salaries of • 
$10,106,604 (difference of AqB-2199- x salary of E2 x 45%). This 
savings of 10.1 million dollars is approximately 48 percent of tKe 
total projectki life cycle savings. Obvious ly,^this should grow 
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Student AOB 
Course Time 
Course Reduction " 



Table 7.4 , 

Cost Saving for FY73 to FY75 

Conventional 
Instruction 

1632 

* ' 196 hrs/ 
. ^ -0- 



Projected Student Load 
Factor for CMI 

Amount of Savings ^ 
in Stu^t Salaries 



4699 
-0- 



2,500* 
104.3 hrs. ' 

t 

46.8 %\ 



10 ^06 ,604 
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considerably as the CMI system moves on into bigger student loads. 
Not only should bigger s^tudejit savings be expected (higher load and i 
,hi^er salaries) , but this illustrates hoy:^ Oil responds flexibly and 
cost, effectively to new manpower requ^rpraents. 

Instructor and. Su p port Pe r sonnel > As presented ^in Table 7.5 , the 
da^a indicates that there has been a significant drop in instructor 
personnel (23 percent). Given tliat these are on the average , E-7 
level, this yields a savings of approximately $1,659, ,408 for FY 75. 
Again, the CMI system: appears to be on target in yielding the cost^ 
benefits appropriate to its development . 

Con^uter Costs / 

It should be noted that for FY 75 and even more so for FY 76 and 
FY 77 there will be additional one-time computer acquisition and 
operating costs (see Table 7.6). Such costs have* to be amortized ovei:^ 
an 8 to 10 year-period in order to .fully reflect their capitalization 
in relation 'to the benefits. Sudi a process, though, does not reflect 
the actual outflow of cash. For the NaA^ CMI system the cost savings 
($10.1^M and $1.2 M) more than cover the $1.6 M current Kor projected 
costs. Computer based training systems always require this early ^high 
Start-up cost bfit within the first four yeai:^^the cro*ssover curves 
with conventional instruction are more than dramatic enough to justify 



suph capital investn^nt. For the' NaA^^ system, the cfbssover benefit ^ 
point has already been reached, n ■ - . ^ 

Hi^r Performance Levels . As Indic&ted in Chapter 3^ the students 
iffider outperform better on the final exam. Therefore, it seems 
reasonable to assmne that \dien they go to the fleet that they will 
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Table 7.5 ^ 
Instructor alnd Support Personnel Savings ^ 



Projected 



CI 



CM 



Instructor 
Levels 

Projected Support 
Level 

Instructof^ and Si^jport 
Level for FY75 

Cost 



Cost Saving 



187% 
(25:1 for 4699) 

292 

(12.1:1 X 4699) 
480 



$4,170 ,240 
(480 X 868y) 



83 

(30:1 for 2500) 
206 

(12.1:1 X 2500). 
289 



$2,510,832 
, (289 X 8688) 

$1,659^,408 



Table 7.6 



Conputer Costs* 



FY 

75 , 

76 

77 

78 

79 

80 



J, C3ost in Millions 

1.6 
2.4 
T» 6.2 
3.7 
2.6 
'2.2 



* Total Cdniputer Costs cover ACP personnel , lease purchase equipment , 
maintenance conmuni cations , and sv^jplies/sxjpport . 
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experience a mre pbsitive\trans£er to on-jthe-jpb tasks J Such con- 
siderations, though p are vfery hard to calculate in dollars as to their 
inpact p .although one )cnows if to be Y'ositive. For our puiposes, it 

can be estimated that 5 percent of the 10. J niillion dollar figure is 

* ■ .* ' 

utilized to derive this benqfit ($550 ,000) . .It should- further be, 

• ' ■\, 

noted that a study of this intact and its real benefit to the Navy 

should be undertaker^ in the future . 

J- I ■ , • 

Student Attrition . Again, the analysis in this area is preliminary 
at best. The rationale for considering attrition: is an awareness that 
CMI produces a significantly lower rate. It could conservatively be . 
estimated at 4.5 percent or more on the average at 7 percent. If, 
ulsing the 7 percent estimate , one takes this figure times the total 
student input fpr a year for each Oil course and then^ assume? that a 
student on t;he average goes halfway through the course before under- 
going attrition, /then one comes up with a figure of $56 ,874 (7% x 2500^ 
X $325) for this cost benq^fit of reduced student attrition.^ 

Jn^ viewing the above cost benefits, one can see $Kat the actual 
CMI system is performing according to the projected levels of cost 
savings and perhaps even in a inore beneficial manner. (See Table 7.7 
for a sunmary of benefits). The more beneficial factor should be ^ 
considered by Navy personnel in trying to assess the full cost benefit 

I -« 

for CKI. In this ivay one is in a*better position to understand tl^^e 

' V' ■ Ik 

fiill iig[>li cat ions of aj.1 benefits. , 

CtohcL^Tent Navy Operational Benefits . In passing, there are two 
additiOTial benefits v^ich should be noted. Th^e are very hard to 



determine costs, and only beginning estimates are provide^ First, 
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liable 7.7 
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— . cm CX)ST BENEFITS , ' 

Goal 1 Reduced Course Time ^(46.8-%) 

Goa; 2 Induced Student ^to Instructor Ratios (23%) . 

Additionjpfl/i Reduced student to Support Personnel Ratio 

Impact "'^ ' , (23%) , ' 

Higher School Performance Levels (5%) " 

Reduced Attrition (21) 

Longer Active Work Tours 

Hiejier Work Performance Levels 

Increased State of Readiness 



/ Potential for Reduction in Manning .^vels- 




* 



all of the sailors going through these schools have finite tours' 
for vMch they have enlisted. there is a 46. 8*" percent reduction in 
training tijne, these number o^ weeks dre added on<'to their active duty 
tour. Such additional tirne provides additional experience for i more 
knowledgeable > more qon^etent^ sailor to be performing in the fleet. 
Such additional time should not be ignoi^^ but actually estimated in 
terms of it^ contribution, and included in the benefits of the CMI ^ 
systm. / One crudely estimate that i1: is approximately half ^of the 
student's salaried savings four^^ witl^in the CMI course, namely $5.5 
Milton or some equivalent additional benefit due to increased / 
competency not salary. ' ^ 

■'' , . V ^ ■ ^'j-' 

*^ Even more importantly, the increased conpeten'cy due to longer 
work tours as well as higher competence levfels yields a pprformer who 
undoubtedly a3ntributes more to the readiness of any giwn ship. Such 
readiness permits a reduction in the manning levels for the operational 
Navy. It is coitmon to heiar that many of the career fields vrfijch are 
being trained via CMI are undermanned by qffiGial^ levels. Given that 
' this is truly the, case ^ it indicates that CMI may be cprit^ibuting to 
an aptual evolving reduction in manning levels for the operational 
Navy. Again, a benefit.. of this type is hard to estimate and the Navy 
ought to^Stvady this possible bene.fit. ^ 

Conclusions " \ . * , ; 

■■ — ' } 

1.. .The Na^i^y CM system is a highly cost beneficial operation 
vAich is saving .the Navy during FY 75 in excess of $10.2 million 
annually ($l(^a |l.*7 - $1 .6) . Such ^ayCSItribution is ^l^ly due to 
. the effect of the Ofl system in comparison to conventional instructiofi. 



Obviously the costs for the Navy to return to conventional instruction 
would be 6noTnious, borxlering on the inpossible , . , " 

2. The Navy CMI system yields significant CQst beneficial contri- 
btSi^ians according to all four, of its objectives, namely: course time 
savings, reducsed instructor st^iport. personnel, higher end of course 
perfoimance levels , and reduced student attrition. Each of these, if 
considered, increase the cost savings in excess of $10,2 million for 

■"•I^y 75. ^ ' • 

3. vJhe projected cost savings for the NaVy Oil system as prepared 
in FY 73 appear to be conservative at this time and even greater gains 
can be achieved. . ^ 

4. The Navy training manager shdpld consider (1) studies of other 
concurrent beneficial effects such as "that attributable to a higher /' 
perfomanc^ level at the end of CMI course and the inpact of this 
factor^on transference to fleet operations ; (2) reduced attrition and , 
consequently, higher savings for manpower deployment; (3) increased ^ 
work-tiai|Bs f^r fir^t £our personnel being^ processed through OJI 
schools ; and (4) potential for reducing manning levels given increased 
coi^Detenby levels .due to this inproved foifm of training. 
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CHAPTER 8^ Urge Conputer Based Training Systems 

* *- In -the training world, it is a cofimon practice to conpare techno - 
logically based systems whose purposes are modestly similar vath the A 
domain, ^ch. con5)arisons and contrasts, p£ course, ^tend to minimize 
the differences in stated purposes, objectives , characteristics , .equip- - 
ment, ipaterials, etc. More importantly , .they minimize ^the differential 
^3nounts of funded resource levels. Finally,- they ^lack the sejisitivity 
oo^nceining the institutional context into whi^ these computer based , 
systems are implemented. On the other hand, a non- detrimental con?3kri- 
son provides for a clear identification of the alternatives open^to 
large computer basfed training systems. Such a panorama of alternatives 
is essential from .a training poinj:^of%iew if the bertef icial" relation - 
ships between cdnqputer systems design and the resulting- training outcomes 
are to be studied. From an institutional point of view, it is foolish 
to think that the economics of the computer based training system are i 
not a prime determinant of its current and future implementation • 
Consequently p the Navy CMI system shall be placed in context with the 
only two other existing large computer based training systems ; this 
chapter is not intended to criticize or make conpetitive comparisons 
but tather to identify the alternatives and illuminate the role of the 
Navy CMI system w^^ the domain. - ,o , 

In the world of computer based training systems, especially those 
designed for large groups (in excess of 1,000 students) there are 
three vMch can be brought somev^at into the same contextual view. 
jgTiere.are large ccnnputer based inst:|^ctional systems in the public 
sector, e.g., the Chicago Publid School Drill and Praietice System with 
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800 pl^us terroinals, and iiv the incUistrial training world, e?.^. , the IBM 
cusltaner engineering CMI system. with 180 terminals, that could be 
described. The ^act that these and other medium. sized systems artre not 
^^^^ *""aS5crihe4 is due^^ot to their inqjortance or future potential, but to 
their overlap with the PLATO and^ AIS.) ^ The systeifis to be 
reviewed /are the Ihiversity of^^IUin^is^ PLATO Computer Assisted ^ 
Instruction System, the Air Force Ady:;^ced Instructional System, Lowry 
Air Force Base and the Navy CMI system. In order to establish the 
context and identify the alternatives, the follpwing Questions were 
investigated utilizing interview techniques as well as a study of 
documents. « Xin all- cases personnel f ron^^ the Uhiversity of Illinois 
PLATO system and from the Air Foyce AIS haVe reviewed the 
write-ups 9nd reacted to them in terras of their accuiacy and represent^- 
tion.O 

1. What are your current objectives? 

2. What is your current conputer and terminal configuration? 

3. How many terminals are operational in the field? , 

4. How many terminals can.be on-line at a given time? 

5. How many courses are on-line at a given time? 

6. What is your current maximum number of students on a given day? 

7. Howgnnany terminals do you project for 1980? 

8. What is. your mean systems response time? 

9. llow many transactions per terminal per minute do you normally 
observe? 

10. What is the character size of a typical course? 
11 • How long is a lesson and a course on the average? 
12. What procedures do authors use to inplemertt a course? 
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13 • H6v7 many authors currently 'are active? ' . ^ 



14. What type of instructional^ model (total displacanent of 
% instructor, supplenental , or impact) do you have for your 

classrooms? 

15. .What effect i$ your system having. on yout institution? 

16. What institutional changes are facilitated? 

17. What institutional changes inhibited? - 
18,. What is the current cost of your terminal? 

19. IVhat is the cost of** your conputer configuration? 

20. What is the coat of your coimamication multiple^ equipment? 



21. What will the operating system and language 



\:ostJ 
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22. What is your authoring to lesson hourly ratio? V (This query 
assumes the full ISD process through computer iraEdementation • 



Unfortunately, the estimate may be ttimcated.) \^\v 
23. Is there an instructor reduction planned for the syst^ and 



institution? ^ ' 



24 . What reduction in course tin© are you subserving? 



25. What are the unique features of your system, and irfiat are its\ 
greatest benefits? 

For the purpose of commmication ^ the answers to these questions will 

be found in the following narratives. A narrative descriptive style 

is being used in order to ^promote understanding and minimize any 

inArldious comparison or focusing on one coniparative element to the 

exclusion of all others. 



The PLATO IV Ctorputer- Based Educ ati on System 

The PLATO sysltan, currently under R 5 D expansion by the 
Co3ipiter-based Education Research Laboratory (CERL) at the University 
of Illinois, is designed to provide high quality technologically-based 
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education at low^ cost. ^ The system was developed iiutially in the 
Coordinated Science Laboratory, ufider the direction Dr: Donald 
Bitzer, A Control Data Corporation Cyber 73-^4 (6700) is the central 
core control . element in Pl^VTO vMch allows for a large number of 
students, at various institutional levels and over a range of disci- 
plines, tp receive instruction simultaneously/ 



Objectives ' 

Although the program objectives for PLATO are not documented at 
present, Dr. Bitzer views them as enconpassing three general domains: 
technical educational , and researdu Underlying all the objectives 
is one ;^eneral goal: to develop low cost, ccmputer based education . 
which will be easily accessible to the general public. The achieve- 
ment of this goal presupposes the attainment of several sub- objectives , 
many of vMch are considered to be unique to the PLATO system* 

(1) The effective use of television chaimels for data distribution . 

' At present PLATO e^nploys a microwave delivery system capable of 
transmitting information to teminals \dthin a 20- mile radius of 
the University of Illinois. The ccHmrnmications cost is now 
estimated to be lower than commercial telephone services. ' 

(2) An increase in teminals per phone line capjacity . Thg dirrent 
PLATO system can serve terminals up to 1200 baud telephone lines. 
It is anticipated that within one year, this capacity will be 

increased so as to accoimodate at least eight terminals per line, 

•ft ' ■ ^. 

(3) Improved technical temina l des ign cost outcomes . The PLATO 

^ ^ • -^^^^ 

system currtehtly utilized a^plasma display terminal L Advancements 

1 * » 
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in engineering technology and an anticipated demand that supports 
mass prpducs^ion should result in costs \>eing reduced by one- fifth* 

(4) An increase in .the processing^mgmory program confijguration . A 
proposed enhancement o£ the OPU/ECS configuration yield? the 
pot^itial o£ reducing cost for servicing a users. 

(5) To develoF/ a nationwide/international neti^^ork . While the present 

o * 

PLATO systf(% is ^ not expected to increase beyond a projected 
• capability of driving approximately 500 terminals^, it is antici- 
pated that other PLATO systene of similar si 2^ will be implemented 
both nationally (e.g., Florida System) and throughout the world. 
Tl^f^se systans will be interconnected via phOTie lines allowing for 
a rapid and cost-effective transfer of both techniques and course- 
work. ^ 
The e<iicational objectives of PLATO, course p are integrally ^ 
related to many* of the technological goals classified above. The most 
basic objective in tile educational domain is to increase the usage of 

■ I , ... 

PLATO by public schools , community colleges , universities , and military 
training systems to approximately 5 ,000 hoi-irs per day . By Way of 
ccHTTparisonp it is estimated that roughly i ,000 hours of instruction 
are being utilized today. * 

The dynamic mission for PLATO dictates the need for an ongoing ^ 
R effort to evaluate the existiiig systen and determine what -changes^ 
might be implemented to reduce costs and enhance instructional capabili- 
ties. Future RmD will^be focused on questions such as: 'Tor v^icH . 
courses does the PLATO system appear most appropriate? ''What different 
types of instructional services ought to be provided?" 'Itow can PLATO be 
made more accessible to the private sector of ^tha country?'' 
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Computer Equipient ^ f 

The display unit used by PLATO is the "plasma display\^anel/* 
vtfiidi consists of two thin% transparent sheets of glass on irfudj are ^ 
placed 512, horizQittal and 512 vertical transparwt conductors. This^ 
technology makes possible graphical display capabilities, selective \ 
write and etase of parts of the display without disturbing the rest 
of the screen, design of symbols , standard alphanumerxcs , animation, 
,and rear-projection of c^or macro images onto the panel. The pl^^ia 
model is considered to be highly cost-effective (current custom price 
is $6500 plus add ons) and well suited to low-cost mass pspductiori. 

The display unit is inconx)rafed into a sophisticated\student 
teijftinal vdiich contains a special key set, a random- actess linage 
selector used for the^rear projection of color micro images , a touch- 
sensitive device, and input /output parts to operate external equipment 
under oonputer control. Among other devices used with the PLATO IV 
teminal is a computer -controlled music box \Aich can be used to 
transmit tunes of varying pitch and duration. The PLATO IV terminal 
is regarded as a highly innovative and technically sophisticated 
device with capabilities that far exceed those of other terminals 
presently in existence . (Some DOD users have found micro image 
proj^ors to be marginally reliable in terms of image placement on 
the screen, e.g. , Trident CBI Study.) — 

The con^xiter to vMch the PLATO* terminals are connected has 65 ,000 
60 -bit words of central memory ,^ two central processors, (CPU's) , several 
disk storage units, and ten peripheral processing units (PPU's) . The 
heart' of the system is'the CDC^ Extended Core Storage unit (ECS). 
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Permanent storage of lesson material is on the disk units , but a 
requested lesson is transferred to the ECS unit where it remains v*iile 

\ 

in use. ^ - 

Current and Projected Utilizations of PLATO 

At present there are between 850 and 900 tetminals participating 
iij the PLATO System. However, it is estimated that the system, as^lt 
nqw stands, can accommodate a maximum of 550 terminals* at one time. 
Approximately 480 terminals are supported now (the. exact number is . 
displayed on all terminals by the system). Given 480 terminals ' 
institutionally distributed^ to schools, and community colleges ^NSF 
sponsore'd - 39 percfent, DOD groups - 11 percent, universities - 10 
percent, and the IMiA^rsity of Illinois # percent, the current ' 
potential for instruction is estimated to be ^800 students (currently 
estimated at 2,000). Approximately 4,000 course hours, encompassing 
miversityT^OTTO^ij^^ college, public school, and military prograns, 
are now on-line . pver 120 disciplines are Represented in the university 
related courses. The exact number of students who use the system is 
unknown, but it is estiimated that on a typical day, there are appro^- 
mately 2,0OQ student users. An average lesson for accredited univer- 
sity courses'^is |ne hour^in'^duratign, v^ereas^or eleinentary instruction 
lessons, range from 15 to 4^3 minutes.^ Each course contains approximately 
30,%K) 6yb\t characters per hour of instruction. Three types of class- 
room models are most/ pervasively enployed — Stpplemental , Impact, and 
DisplacCTient . (The Sqpplemental model is a form of instruction that 
aids a n©re c3onv6nt^ional ongoing instructional purpose; mathematical 
drill and practice in the elementary school is an excellent example of 



such supplement. An Impact application is the plication of ^technology 

to bring* d)out a change in the instructional process and institution 

and enhaiice the beneficial outobraes. The Navy application of coraputer 

^ support of individualized ihstruption is an example of siich ijipact. 

, Total Displacement is the CAI tutorial conception advocated in the 

1960*5 ahd foresaw that most or all of education could take place by 

a technical means as opposed jtp a more conventional human interactive 

^^Tpocess.) Dr. Bitzer feels tliat the moit effective plication of 

\. — . \ 

PLAPnO appears to be with the Impact inodel . 

very desirable feature of PLATO-is the rapidity with which\ 

canprnmicatians are delivered i:o students. The average systems respwise 

/ time is 130jnil^seconds, and roughJ^ 30 transactions per minute are 

^xpecti^. ^ -y • 

Projecting to.the future , the* PLATO group anticipates that by 

30 June t9i^ the number of operative terminals will , increase to 960, 

with the goal being an additional-^creale to 2 ,000 terminals within 

the next few yfears. Although the University of Illinois system is not 

expected to extend in capacity beyond 2,000 terminals, it is^ assumed 

that by the 1980 's similar PLATO systems will be iirplemented throughout 

the world. Thus, it is quy.te possible^ that in 10 years time there , 

■> . . " ' 

could be as pany as 100 ,000 PU^O terminals in operation. These could 
— «^ . • » 

sfervjB literally millions of student^. 

<>» • - 

' ■ ■ .. 

Authoring Procedures - . ^ 

All authoring is perf onned exclusively through the terminal by 
use of the TlJTOR\^iguage . TUTOR can be characterized as a highly 
powerful and efficient CAI language, relatively simple to learn, and 
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rich in its ccmmunicationai (instructional) capabilitiesy^ (Some DOD 
authors have observed that TUTOR is conceRt- oriented anjf limited in 
full file handing abilities.) It is generally assumed that inost 
instructors, with some assistance from CERL consultants and on-line 
lesson-writing-prograns , can acquire considerable fluency in authoring 



courses in ^fairly- short period of time , In fact it has been 
observed that the highest quality materials are produced whenv^the 
instructor themselA^s; rather than professional programmers , (do the 



authoring. There are approximately 1,000 authors, including students, 
currently active in/the^ University of Illiiiois System. ^For^emerienced 

authors, the programming to lesson ^hourly ratio is estimated to be in 

' . • * X \ V. 

^the neigKBorhood of 40-50:1. (|)OIf experiqnce at fTTC, San Diego ; 

indicates th^T^this ranges from 100 to 250 hours V^r lesson hour.) 



Cost Figures 

Estimates of the cost figures for thfe majof.components (if PLATO 
are a^ follows: Display terminal - $2,000 ultimately, currently 
$6 ,500+; Con9)Uter - $13.5 M (Conputer, /s M and terminals $5.5 J 
Language/System - 30 man years (30 man years x $20 = $600 ,000);\and 
Ccknimications/Multiplex equipment - $150 ,000 per year, 

Benefits and Icaplications ■ , ' ^ 

At the present timd| no instructor reduction is foreseen, ats a 
consequence of utilizing PLATO, although ^ significant reduction^, if 
desired, vjould certainly be possible. Instead, the benefits derived 
from the PLATO system are being translated into increases in teacher- 
to-student ratids. Since the PLATO system can deliver instruction to a 
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large nunober of students with relatively little, increase in cost, 

^ ... 
liversity coui^s can e3q)and in number and enrollnient jwithout the 

luirenient of e^lbying additional faculty. The^ potential saving 

injcost is, therefore,, oonsiderabler^ (This reniains to be achieved.)/ 

With respect ^ time savings in learning, little evidence is ^ i 

^^rpsently ^available since the roajoiity of PLATO instructors structure 

their courses according to conventional classro^ schedules. The few 

exanplQS of self-paced courses which have been iiipleniented suggest 

that time saviiigs as great as 33 percent may be realized. 

^ In sunmary, the PJLATO system appears to conprise an effective, and 

relatively iii|ejq)ensive approach to educating large numbers of students 

It encanpaisses many unique features, the wost significant of vAiich, 

according to-Dr. Bitzer are: » ^ 

(1) The TUTOR langiage , vMch is rich in its capabilities althouj^ , 
one observes a wide range of reactions by authors. 

(2) Multi -media Capability , vMch is made possible via the plasma 
screeif terminal. The teiininal is at the heart of the System and 
is its ^greatest benefit. 

(3) Overall^ Phiiosophy toward CA! , vtfiich avoids formal management 
structures and labor differentiation (writers vs. coders) while 
supporting a total systems approach on the part of developers, , 
engineers! administrators, authors, etc. ' 

Air Fbrce Advanced Instructional System ^- ' 

The AIS s}b^^emVgrew out^ a realization that a total systems 
approadi was essentiaJSv to address the conprehensive requirements of 
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Air Force educational and tjechnical trainiftg. The AIS evolved from a 
concept first developed in 1966 by the Instructional Technology group 
at USAF. In 1969, the L<3wiy AFB,B^i[iian Resource Laboratory, 
Technical, Training Division; assuHjed responsibility for this advance 

development program and began the formal justification and planning 

' V ' ' ' ' ' ' 

p. ^ • 

process. Recogniziftg the primary driving force of cost- effectiveness . 
in technicid training, the^AIS.was designed to be primarily ^foperftr"^ 
tidftal conputer supported instructional system (CM) with a powerful*-^ 
R ^ D capability for complex training • The planning effort was . / 
extensive, in that a ^systems engineering approach was follow^ ijx 
oising standard DOD 1379 planning techniques and evolved "^fttough a ^ 
nunfcer^of iterations. For example, the project recognized the necessity 
of segmenting^ the system into sub- system reqp|^jenffints of which there ; 
are seven: (1) instructional materials; (2) inslTuctional\trategies ; 
(3) media; (4) computer software; (5) coputer ^rdware ; • (6) personnel 
and training; and (7) related requirements (for example, reliability, 
maintainability, and human factors) .v 



Objectives . 

,The AIS is an artful mix of operational and research objec- 
tives. , These are reflected in their stated goals a^ well as their 
specific measurable obj.ectives'. The dual goal fdr the AIS was to"^ 
develop a cost-effective computer based instructional systeth for four 
ATC courses ^^tudents AOB = 2100) and to ptovide a facility for the 
exploratiijn^of parameters and research alternatives .that will contribute 
to Air Force training andjeducation. This general goal was furthjipr 



broken dcMi into the following subgoals : ^- 

(1) Adaptjveness > The AIS shall provide procedures for facilitating 
student individual differences according to th6 utilization of 
appropriate training algorithms to specify specific strategies/ 

(2) Flexibility , The AIS sh^l offer a sufficient array of^training 
alternatives so as to provide for both cognitive and performance • 
requirement^ as vrell as individual and multi-person behaviors. 

(3) ' Expanidibility . The AIS is designei^ ^ prDv;i|e,;fd^ variations in 
student flow, number and variety of courses, and varied locations 
^lile stqpporting individualized approaches that speci^r prescriptive 
learning ad^tive testing, evaluation, and cost optiMzation. 

(43 Modularity , The AIS is designed to provide for both the prepara-. 
tion and revision of cpurse materials so as to mpre adequately meet 
changing trainirijg requirements, . 

(5) Cost Effectiveness , The AIS shall demonstrate significant 

■ '■ ■ ^ ^ . ' ' ■ 

reductions in coinrse length, stable (no increase) elimination rat;jes , 

and improved end of course performance. All of the above goals pre- 

1 ^ ■ 

stqjpose the availability of a rich array of off-line media alternative^ 
as veil as the necessary coiiputer sipport. 

The above goals have been translated into more specific objectives* 

. • ...^ ^ . . ■ . 

These give a mote measurable array of progress" vtfiich the AIS can 
pursue. ' 

1.^ The AIS shall of f»r a significant cost savings . A 25 percent 
reduction in course length and an appropriate reduction in instructional 
Staff and resource ^as well as the future c^ability to provide for the 
Ml-vol^teer force, reduced manpower, and e^qpansion capabilities. 
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2. The MS shall demonstrate the ability tp trafai students 
to the Afe specialty standards and according to ATC evaluatiohal 
procedures > \ 

3, The MS shall be interfaced to the non-training pai 




of the training environment , for example , recruitnteiit persartnel\ and 
base facilities* ^ 



Th e jsys tern shall ^be usable by jp^r T8r(^ perspnnel' as a^ tool 
for converting other courses to ^presentation on the AIS > 

.5. The MS shall be e^yandable or replicable to meet ia ^ 
vdde s^ctrum of Mr Force training and educational requirements > 

6. The MS shall collect training data to form the basis " 
for its own^evalu^ation and inyroyement ^ 

7 . The system ghall be capable of supporting exploratojy efforts 
to develop and evaluate new .training technologies , 

Any of the above objectives could be eDq)anded at great length. 
Each is being pursued diligently by the Mr Force through its AF HRL>(Tr 
Division, ATC personnel, and supporting Mr Force units,, as well as a 
$10.8 million contract with the McDonnel Douglas Corporation (as wiLl 
be explained, the $10.^ million provides^ for the acquisition of the 
computer and media hardware jas well as the development of significant 
software md wide -spread course conversioil) 

. X ' . ■ \ 

Computer Equiptoxt , , V 

The'tanputer is ,a CDC Cyber 74 with 65 ,000 60-bit words of central 
memory, -500 K words of extended core storage, 10 peripheral processing 
units (FPU's), several disk storage units, standard input-output devices. 
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aticta flexible coiranunications system (telephone and video links),,. The 
initial prototype shall have nine to 16 (M terminals vdth enough- 
conputer Capability to siq3port vp to 50 Oil 'terminals- vdtiiout system ' 
response dela>^. A CMI, terminal consists of an optical mark reading 
device capable of inputting standard test scorii^^i^ts ; and a. 240 
character/sec. chemical heat printer iSlus a mini -computer controller.^ 
Accordginjg to the design plan^there shall be about 50 to 75 plasma 
panel interactive terminals having the general characteristics cited 
in the -PLATO descrii)tion except that a n^ -rear projection method is 
yet to be inplemented. ^ 

, From a conmuter hardware point of view the AIS is essentially 
equivalent to^ but Slightly scaled down ^r^^ the Uhiversity.of Illinois 
PIATX3 configuration. Obviously, there are significant differences, ^ 
however, in the software given that the AIS siq>ports Ott teitniiials ; 
'"^^is repipesents a highly significant difference in th^. systems. Given 
the likelihood of an extended future , one can also anticipate that 
various experimental simulator devices will also be incorporated within 
the system, depending on R § D outcomes. 
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Current'^ and Projected OtiXization of the AIS 

At present the AIS is still in its initial iinplementatipn phase. 
The majority of the Inventory Management/l^terials Facilities course, 
vMch represents nearly 50 percent of the AI3 student load ^ proceeding 
iSi Instructor Managed Instruction (I^n)/mode. ■ There is now a limited 
on-line crtiputer si?)port of test Scoring, ho«^Ver, a\l other test data ; 
is batch* entered into a data base reflecting the learning 4^(1 perfor- 
mance levels of the students. This is entered via usual 
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procedures. Thed|lans call for rapid application of computer snjport 

/ F • 

•during ^e early portion of the coming fiscal year. Both the applica- 
tion of adaptive testing and adaptive instructional modeling shall 
receive the preliminary feasibility evaluations during the first half, 
of the fiscal yfear. 

The prototype system shall support 700 students vdthin a given 
6 hour shift or 2100 students daily. The average lesson shall vaiy 
from about 15 minutes ^ to three hours with a mean approximating 60 
minutes. These estimates may change as greater experience is gained ^ 
concerning the actual application of CMI procedures. At this time it 
is prCTiature to estimate the number of characters per hour of instruc- 
tion within any given course. As to classroom models it is anticipated 
that the AIS will be a mixture of the inpact and displacement models. 
It is ijnpact in that it changes the AF institutional training processes. 
A transition*^ plan that incorporates change procedures is a specified 
requirement. It will be displacentent in that significant functions 
like cognitive learning and testing shall be conpletely under computer 
control. 

The operating characteristics of the AIS will undoubtedly be highly 
similar to that of PLATO. It is anticipated that the average system 
response time shall he less than 250 milliseconds for interactive 
prograns.^ An average of 2.0 transactions per minute on the CMI 
j:eTminals and 60 transactions per minute on the CAI terminals shall 
be expected. In terms of the future, the Air Force is curia^ntly 

^ ' :\ f ' 

studying the requirements for expanding the current AIS to si4)port 

■ V 

instructional development at a number of bases throughout the oonti- 
nental.United States. In addition, one can anticipate a serious 



113 



evaluation of its jqjplicability and dissemination to all of the tech-- 
nical training centers currently operating within the Air Force (there 
are five of these in ninnber) . Therefore, one can anticipate that the 
AIS could be disseminated broadly throughout the Air Force as well as 
the DOD training world. For the 1980 's the conputer system has the 
capacity to e^and to several hundred CMI terminals and several hundred 
interactive teiminals (given an upgrade in the hardware) . Initial 
on-line usage of Oil terminals shows that about 120 transactions per 
hour can be handled per terminal . The actual number of students 
handled per terminal per day depends on the interactive rate built 
into the curriculisn , 



CAMTL/Authoring Procedures 

TTte i^jbr effort withi^ft'^the 'AIS project is the development of a 
new languag^ capable of. aciiieving a number of highly operational 
'objectives. \First^|aui foremost, the language is being designed and 
implemented t(\be of easy lise and applicability to instructional 
systems developn^t and operation. Second, the language represents 
the application the latest concepts in conqDUt^ng science. For 
example, CAMIL stal^eraents represent a natural language, sentence- like 

■A • \ . . « - 

' structure?. > Each serttence in turn, can allow for embedded sentences, 
-similar to those found in the English language. At the same time all 
of the power and capabilities of the operators found in such languages 
as APL, Fortran and Cobol are present. An additional software feature 
is an iritegrated file handling system vMch proAddes an on-line 
management information system (MIS) to support adaptive instruction. 
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This should allow the 041 course managers to adjust parameters and 
floi^ vdthin the course to achieve additional ^efficiencies and enhanced 
leaming outcomes. ^ Since the languajge is still in its prototype 
operational stages, many of the Ccsnmon estimates, cannot be made. 
For example , it is inappropriate to estimate Kow many ^/authors are 
presently" using the language or wfiat the programming to lesson hourly 
ratio seems to be. . - 



( 



Cost Figures . 

Many components arc/ still under acquisition and development so 
that the follqwinjl cost figures are a preliminlbry estimate: QMI 
terminal - $18,000; CAL terminal - $9 ,200 : conq}Uter - $1.9 million; 
software ^velopment (30 man years x $50 ,000 = $1.5 million) , and 
coranunications equijpment >^ $150 ^000 . It should be pointed out that 
the above cit^d costs represent an indeteiminate mixture of operational 
requirements and research. IJhdoubtedly , as the AIS matures and is 
di5;se:ninated, a much lower capital expenditure would yielded. 

Benefits aiKi Inplicatians ^ 

Similar to the PLATD system at this time, it is d^ficult to esti- 
mate the instructor reduction possible through the AIS. On the other 
hand, the goal of a 25 percent reduction in course length is being 
achieved using the overall technology of the systein. (A 40 percent 
reduction is presently being achieved in the Inventory Management/ 
Materials Facilities course.) . ; 

In sunwary, the AIS is in the early stage of .its inplementation. 
Its full benefits and potential are yet to be determined. This system 
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could easily prove to be the most optimal approach to the^ overall 
training in the DOD enviroiS^t in that the systCTi allows for an 
appropriate mixture of (KT and^CAI capability. Such a mixture is 
an obvijpus requireitent if one oonsijders the conplexity found wi^n 
technical training, especially as it advances toward the actual i^apons 
System training stage (experience vdth the S3A System indicates that 
CAI is a highly beneficial ^simulation training approadi) . 

Navy CMI System ^ ' . 

Since this moiiograph has eodiaustively described the Navy On 
system in terras of its past^ present, and future , this section shall 
only provide a sumnary; The purpose of the sxjmnary is to answer the 
above stated questions and provide the reader with some understanding 
of the differences of purpose and level of development. 

The J^avy CMI system has been designed to enhance the dissemination 
of individualized instruction throughout the Navy technical training 
world/ The primary thrust is individualized instruction; CMI is a ' 
supporting system to^^this rapid dissemination. In this regard, CMI 
can be considered an inpact strategy in that it has allowed for the 
rapid change of miany of the technical schools within the Navy community. 



Objectives ^ « 

As cited in prior chapters the Navy system has fpur objectives: 
(1) a 36 percent reduction in cours® length; (2) a 20 percent reduction 
in instructor and stpporting' staff personnel; (3) an inprovement in 
student perfonnance on end* of course. exam;inati6ns and (4) a reduction 
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in within course student attrition. As cited in Chapters 3 and 7, the^ 
Navy is currently achieving all of these objectives.. In addition, the 
Navy is OHnmitted to enhancing the system in utilizing any instructional 
strategy that effectively contributes to the above four stated objec- 
tives* Therefore, one can anticipate the evolution of new instructional 
strategies to achieve even greater benefits duriiig the coming years 
(see Chapter- 9) . * ' 



Confuting Equipment „ . \ * 

The Navy has ^a Hcne>'well Sei^ies 60 conputer with two CRU having 
13XK-36, bit word cores. Eight large disks (117 M characters each), 
six tape drives, and state o£ the airt 1/0 components make tp the 
configuration. The multiplexors are mini coin)uters capable of local 
support, line switch, and telephone oriented transmission. The design 
msaximijes system reliability. ' . ' / 

The 04I terminal is composed of 120 CPS keyboard/printer and an 
Opscan 300 sheets /min optical reader. The administrative^ terminal is 
composed of a micro processor/message switcher (with a disk) , a 300 
LPM printer, and a 600 CPM card reader. 

As to cost, the GSA schedule for Honeyyell can be consulted and 
it indicates an approximate 30 percent disci^unt. The estimated hard- 
ware cost for lease purchase is approximately $8 M for the six year 
life cycle. 



Current and Projected Utilization of the Navy CMI System 

The Navy system is planned to have nearly 17,000 students parti- 
cipating oxy a daily basis by 1980. As indicated in Chapter 7, this 
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represents approximately ZO-*- courses.* Obviously, the Navy has selected 
its. highest student flowSt^rses far inclusion in this activity, and 



one can anticipate that as it :l(iatuTes even some of the medium flow 
courses cartvalso be ^incorporated. . ^Analysis of the ai)ove equipment 
indicates that it (Sould grow substantially beyond the t^irgetted level . 

In reference to lessons, the current (M system encodes the 
grading of tests and the next prescriptions. Typiqally these represent 
about 3,000 characters per lesson.. The system response time shall have 
a mean equal to or less than 30 seconds • It is anticipated that two 
transactions >per minute shall tak^ place at each Oil terminal s 

Authoring Procedures ' ^ 

Authoring procedures^ are an off-line activity with input taking 
place by cards . As indicated in Ch^qpter 4 , the language is highly 
specific and requires a comprehensiine ori^ent^tion- to the systOTi in 
order to be able to cope. Fbrtunately, since the majority of thc^ 
leamii^^materials are off-line, specialized individuals can do coding 
in a very rapid manner • It is currently estimated that the program- 
ming to lesson hourly ratio is approximately 30 to 1. (This figure 
includes textual media conversion as well as computer activity.) 

Cost Figures 

* The following estimates are provided for the componients for the 
Navy Oil system: CMI terminal - $9 ,000 fconputef - $2.5 million; * 
language - 5 man years x 40 ,000 = $200 ,000 ; computer and multiplex 
equipment - $300,000. ^ 

a 
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The Navy systgffi, as illustrated in Chapter 7 , represents a highly 
beneficial ^^roach from a cost point of view. In addition , the 
performance data indicates that it has an acceptable to enhanced level 
of effectiveness. The desired reduction in teacher to student ratios 
of *20 percent has taken place . The targetted 3(J petcent reduction in 
course time has been exceeded and presently resides at 46.8 percent. 
It is anticipated, though, that this figure will drop, slightly as more 
courses are brought aboard. 

In sjummaryp the Navy CM system appears to liave many idvantages 
.from an operational point of view. Given its background and growth 
frdn an R^D effort,. it has allowed for appropriate expansion and* 
dissemination" throughout the Navy technical training «^or Id . This 
process will have to continue over the next few years for the -ultimate 
benefits to be achieved. 

ConclusioriS 

1. Each of the three coirputer based systems described have inpor- 
tant and unique purposes, go^ls, and pplementation characteristics. . 
In essence, both the operational characteristics and-^n5)onent cost 
factors clearly document that each of these three^ systems should be 
continued and monitored in terms of their ultimate contributions to 

V 

large training requirements. 

2. The Navy CMI systqn represents on^ encl of the continuum vAiich 
maximizes on the training of large numbers, in fact, eight times as 
large as the nearest system and has the potential to support even / 
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larger nundbers. Given the requirements for cost savings through 
centralization, it seems only natural to watch the expansion of the 
Navy system towards its goal of nearly 17 ,000 students and on upwards 
to two or three times that number. Such an approach, pf course , may 
require the consi^ration of tri-service sponsorship of the system and 
^'gqjplication. , " . 

3. Given the cost factors and current savings, it seems appropriate 
for the Navy CM system, to consider exploring further sophisticatflins 
in its training strategies in order to thoroughly explore the full 
boundaries of its potential inpact and cost effectiveness. 




/ 

■5 



1^0 



120 



(iiAPTER 9 . Future Development jFor Navy CMI ^ 

The purpose o£ this cliapter is t6 explore some of. the possible 
new instructional* strategy alternatives open to the Navy 0^ System. 
The purpose of this exploration is to identify training alternatives 
which Mould enhance the cost/beneficitfl impact of the Navy CMI system. 
Obviously, any alternative which cannot be docunented in at least an 
advanced development sense, or has a high likelihood of inproving tlie 



current CMI system^ should be held in abeyance until further evidence 
indicates its preliminary implementation or rejection. 

Given the success ^ to date of the Navy CMI system iri terms of 
achieving its objectives and its costA>eneficial outcomes, one could 
r^se the question, ^TVhy implement new alternatives?" Thefe are two 
very persuasive reasons. First, technological* systems tend to over- 
achieve themselves at times. For those that are creating a mjor 
breakthrough, even fui/ther benefits can be achieved by appropriate 
enhancements of their approaches and the domains of activities they 
enter. For Navy CI1I this would indicate a further investigation of 
not only enhfliicing instructional strategies, but also the kinds and 
types of course it supports. Secondly,^ it is conmon^for tecnnologi- 
cally- based systems to undergo repeated cytling between research and 
on-going operations^ The researdj^ and development activities act as 
a stimulus to further fine-tune the CMI Systems. Given the amount of 
DOD, money bein g current ly .invested in basic research (6.1) and advanced 
development (6.^) , especially in the area of computer-based training, 
it would seem, ill advised not to continually survey the outcomes of 
this effort and consideip the possible incorporation of some of these 
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developmeiits within the Navy 04I system. Obviously, consideration of 

.these alternatives should be determined by viiether or not they meet an 

' • • . 

appropriate set of criteria. . 

» .. ^ - 

The consideration of criteria for preliminaiy implementation of 
a new instructional strategy or as;?ociated training technique vdthin 
the CMI syst^m^ifequires extensive analysis, assessment, and policy 
determination as tdf their inpact on, the on-going operation. . The 
following criteri^^ seem appropriate for the consideration of new 
alternatives: ^ ; ^ / , 

1. The new training alt ernatives should represent a ^significant 
contribution to tl|e reduction iil cost and the enhancement of the 



training outcome^ for th e Q^H sys tem 1 ' 

2. The new altenigtives shc|uld not disturb the on-going operation 
or make excegsiyig rydificatj ^qn^i ^ but rather should fit 

into the on-gping ppeyation in tig form of an extension or further » 
enhancement/of th^ on- going confuting system . ^ 

^ 3., The new training alternatives should have a sound experimental 



base; therefore, the connectio ns wit h the Navy* s ba^ic research capa- 
— - \ 1 , ^ ^ ~ 

bilities should be^ obvious and direc t. ^ 

4. The jew alternati^s shou l d be consistent with and sup portive 
of the mission- of Navy, graining > 

Given these or more refined criteria ^ two major trends of reseailrch 
prototypes are appropriate for consideration by the Navy CMI system. 
The first of these rfelates to the enhancement of the instructional 
strategy process;' the second relates to sipporting the institutional 
ISD process amd personnel. Each of these shall be considered in turn, . 
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For each considered trend (instrtictional strategies and ISD 
processes), there are cumulative interrelationships at each stage. 
For exanple^ each stage assumes the existence of its paralle^^^-^ement 
and shares data and, findings. Second, each vdll require eonput?r soft- ' 
ware enhancement that will stppdrt each in is common manner. Finally, 
the eight considered R^D elements mutually support .-each other in a 
fashion that should lead to a vastly enhanced CM system. (Obviously, 
each considered R^D element qualifies according to the four. above 

u • 

criteria; each would require planning and detailed designs to be 
in4)Iem8nted.) 

9.1 Instructional Strategies 

Instructional' strategies can be defined as the development of 
training resources so as to appropriately create a sequence and 
environment for a given student so that his learning and performance, 
is maximized. From this frame of reference, there are four obvious 
stages throu^i^ich the Of[ system could appropriately develop in' 
tenns of its enhancement and growing sophistication. These ar^Ss 
follows: (1) operational research to enhance individualized learning^ 
(2) adaptive testing to improve the measurement process vAiile reducing 
testing time , (3) adaptive management of the system so as to dynamically 
match resources with student requirements, and (4) complex training 
through simulation. Each of these will be discussed in turn. 

1. Operational Research to Enhance Individualized Lgaming . The 
application of operational research techniques plus th6 developinent of 
operational learning feedback mechanisms have proven powerJ^l in both 
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Figure 9.1: The Interface of ' 
R^D with the CMI System 
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management and training systems. While the CML system has the existing . 
' capabilities to provide masses of data potentially useful to LCS's, 
cx)urse manafgers, and instructionsil designers, there appears to be an 
unfortunate limitation in .the instructional diagnostic reporting capa- 
I bility. This' deficiency may be largely'^due to an emphasis on the 
- individual sttident in his everyday instructional progress to the exci^^* 
usA^E-^ sion of learning characteristics of groups of students within the J 
\ syfetem. Thus, the available data are not systematically stored or 

organized so they m^ be retrieved with their full interrel^ation- 

I 

^hips and inplications/ For exanfple>, performance times are accessible, 
but without reference to the categories of students or specific 
.'segments of content.^ While these data are available on an indiAridual 
basis, no provision has been made for grotp or system inpli cations Vso 
. ' ' ■ as t(^ monitor grotq) progress and fluctuations over periods of tim^,, ^ 
'^y Moreover, the current data do not adequately address the characteristics' 
of- levels of'^difficiiity of lesson material , appropriate assignnjent of 
media, and the effects of ^mediation. Therefore, an operational'^ 
^ research thrust should be pursued which on the one- hand, atten5)t5 to 
gain more i^omiation bn grotps of students and at the same tLiic looks " 
at inplications for significant instructional strategy elements. This 
is the main focus of this initial stage. 

Within the DOD training research world, there is extensive evidence 
to indicate that> assessment of fui^^tional reading, learning styles, 
instructional adaptation indices , information processing indices , and 
incentives can provide significant insights into individual arid group 
behaviors, both from current progress 'and predictive points of view. 
It is proposed that these existing tests be given to sufficient 'size * 
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saople of Navy staadents^ and tiKiBn be related to dati^ concerning the 
level of learning difficulty, assignment of mBdJa>: and remediation. ' 
Siirply, an individual, by ^143, by instructioiial strategy data cube 

. \,. . ■ ^ ■ . ^ 

should be formed vrtiich shall allow for better insight into the assign- 
nient of students, rather than continuing the voluntary student self - 
selection strategy currently found in Navy tCMI. Such a system of 
, assignment has yielded enhanced perfbrmance in the 10 to 20 percent 
range. While^ there is lilidted evidence concerning reduQtions in course 
learning time, there is an obvious relationship, and one can* anticipate 
at least a five percent time savings. Therefore, this operational , 
research stage appears to have great benefits ancnshould be aggressively 
pursued. 

-^2 . Adaptive Testing to Improve the Measurement Process While 

Reducing Testing. Time . V One of the key component^ vdthin Navy 
CM is the systems testing, of the/ students' pcfcfoiroance ipon conpletion 
0% each module, time demoted to this activity varies between 18 
and 25 percent of the total time spent by a studehti in a course. Given 
such an extensive coimdtment te^roeasureTOnt, the application of adap- 
tive testing techniques appears appropriate. Adaptive testing, 
especially ilien inplemented over an individual teminal, has yielded 
better than 50 percent reduction in testing time. Given the nature of 
the equipment, it is proposed that a pilot study investigate the ' j 
application of adaptive tes;^ing techniques to ,the existing batch 
oriented CMI texroinals as opposed to individual CRT terminals (it 
should be noted that no enhancement of equipment would be required in 
order to pursue such a pilot study) . If it can be documented that 
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batch- adaptive testing is equivalently effective in terms of time 

saving^, such a procedure could be implemented with little or no cost 

7 

to the current system. Such an assertion as. this is possible in that 
the actual procedure for assigning tests can be adjusted to adaptive 
techniques with little eJ^Fort. 

3. Adaptive Managenent^of^ Systan so as to Ij»aiamically Match 
Resources with Studen^ Requirements . Adaptive management 
refers to the twofold application of optimal allocation of resources, 
such as learning center supervisors' time or laboratory equipment, 
according to the individual and group parameters found for the students, 
\ At the present time, the Navy CMI system presents the same^pres^iT^tio 
to all students, given their equivalent progress through the course. 
It is proposed that further individualization couldlake place, while 
pre^minary research la. still underway (NAVPERSRANDCEN/MSU Study of 
Adaptive Instruction). In addition, a dynamic scheduler for criticfilly 
costly resources , be these hunan instructors or simulators , can insure 
better utfilization of these resource? within the Navy training environ- 
ment. -it-4s on this ba^is that, it; is recomnended that^bie Navy CMI 
system pursu^this line of developn^nt along wfOi the other two 
branches of the service (this is an\ active part of the Air Force AIS 
progfram and the Amy CTS System) . 



4* Conplex Traijiing^rou g h SiJnulaj LiOTi. During the past decade 



and oner half conputer driven simulators have become a common phenomena 
in all forms 6f military training. Unfortunately, the cost of a 
weapons system-like simulator, in many cases, exceeds that of the 
weapons system itself. Therefore , it seems appropriate to enhance 
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the Navy Oil system in the long term so as to cperationalize those 
aspects of simulated .training appropriate as a precursor or follow-up 
to the availability of the actual equipment. Simulated training on 
the S-3A or in the preliminary TRIDENT training study indicates the 
power and cost effectiveness of such a CBI appioach. •Given that this ' 
is the fourth stage in the trend, it is anticipated that further 
results over the next two or three years will contribute significantly 
do that once the Navy Oil system embarks on such ^ endeavor, it will 
firmly proceed based ipon training research findings . 

-* I - 

9.2 Institutional ISP Processes 

The interrelationsliips between CMI and Instructional Systems 
Development (ISD) presses are both extensive and in many respects 
undocunented and/or unexplored. The goal of this second trend of 
reseaCrch will be the clarification and the eirpirical evaluation jof 
the ISD processes as they interface with QAl requirements . .Thii 
effort should lead to a better definition of the actual interface 
reqidrements and provide clearer cost effective techniques and 
methodologies for facilitating investigations of this critica!i^^-^ 
developmental area. Given an understanding of the current Gp "system 
and its likely developments in the future, the following foUr stages 
are offered within this ISD developmental trend as follow^: 
<J ISD model evaluation for (Ml requirements 



... 



2> Testing of the cost effectiveness of CMI elements 
J. Adaptive interface of the ISD - instructional process 
4. Computer based ISD de|i^lopment 
It should be pointed out again that each of these stages not only 
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contributes sequentially, but are interfaced with the instructional 
strateg)' for parallel stages , that is , the operational research will 
feed in directly into the ISD model evaluation and QII requirements . 
In turn, adaptive testing provides theHbasis for fine-grained analysis 
of the CMI cost effectiveness. Adaptive management leads directly 
into a consideration of adaptive^ISD instructional interfacing. 
Finally, training simulations^re of the same complexity and soj^histi- 

—f" ■ 

ISD Nfcdel Evaluation for (MI Requirements 

IXaring the last five years. Navy training has created, and propa- 
gated significantly new instructional systems development models for 
the creation and updating of curriculiin and associated training 

" 1 

materials. These models profited extensively from/prior research work 
in task analysis. From the beginning the Navy CMI project has inter- 
faced' and utilized Navy ISD personnel for the conversion and develop- 
meht of CMI courses. The requirement for building a bigger pool of 
expert manpower is obvious and critical. At the same time ISD models 
have remained relatively unevaluated in terms of each component's 
output (it should be noted that the overall effect 03f the ISD model 
> process has proven to be highly effective) . When two major systems , 
namely, ISD and CM are required to interface, it seems appropriate 
to eva|^te<=the ISD model in terms of its contribution to this CMI 
effort. The focus of this. first stage of research study is to empiri- 
cally docunent the contributions of Navy technical training ISD aiid 
courses as ^hey undergo conversiom. This documentation will be 'in 
terns. of the manpower employed, the procedures utilized, -and their 
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ultimate effectiveness within^ the CMJ operation. Utilizing data from 

* . ^ ■ 

the operational research stage in the instructional strategy trend as 
well as specially collected data, one can anticipate that those ISD 
conponents most critical. to the CMI requirements .can be identified* 

Testing of the Cost Effectiveness of Qjl Elements 

As reflected in thi^monograph, the cost effectiveness of Navy 
CMI is outstanding. Unfortunately, tKe contribution of various compo- 
nents renains to he established; for exan?)le, is it the ISD process 
that is contributing the most to this effectiveness , or is it the 
testing 3nd prescriptive process? * One hears many advpcate the student 
tracking and projection incentljve techniques as being at the heart of 
the inpact of the CMI system. The thrust of this endeavor shall be to 
utilize data from operational research and adaptive testing in assessing 
. the effectiveness and contributions of each of the CMI elements. \ 
Critical base-line data shall be utilized as the instructional strate- 
gies evolve so that component effects can be isolated. ^ It will also 
be inportant in terms of the - relationship between ISD personnel and 
instructional personnel, our next research stage. 

Adaptive Interfacing of the ISD Instnifctional Process 

Chapters 5 and 6 spoke to the specific functions and relationships 
of both the Learning Center Supervisors and the ISD personnel . Unfor- 
tunately, the interrelationships between these two groins have been 
infoimal. The exact coirnnunication and purpose of the conmunidation is 
yet to be docunented. For example , should the ISD personnel be 
paring materials to be utilized by the Learning Center Supervisors as 



they yndertake new functional roles? I£ the LCS's \vere given even 
broader ailtematives for remedial tutoring, personal counseling, and 
application of human resource concepts within the context of technical 
training, v*iat types of materials ^should be prepared by ISD people? 
In essence, the focus of this study is to identify af:g0S?pTiate rela- 
tionships between ISD personnel and the Learning Center SupeArisors. 
This mutually facilitating interrelationship will be' tested by having 
new functions evolve for selective, lead LCS's and evaluated in terms 
of the support of the ISD group as well as the LCS's abilities to 
accept and expand their functions . The intent is not to Vedefine 
roles but to enhance them. This is predicated on the fact that_each 
of the groiqjs expressed, to varying degrees the LCS's more so; the 
ISP personnel less so - their worth or their, significance within their 
activities . By enhancing their roles and applying a proper utilizatioi> 
of leadership managen^nt techniques , one can anticipate significant 
increases in effectiveness. 

Conputer Based ISD Jtevelopments 

During the past decade^ the utilization of a. conq^uter for curri- 
culim development has been proposed by a number of leaders and groups/ 
The preliminary attempts at such places as Florida State University 
and Stanford University have proven to be qiiite pfbmising. Within the 
, DOD conirajnity this activity has been limitedly e^splored at NPRDC* 
^ Therefore, it seems ftost approj^riate to propose it as a culminating 
sophistication for tlie Navy CMI ISD trend. Miat is being proposed is 
the utilization of computer terminals for tlie actual planning, 



authoring, pilot testing, and foTmative evaluation of materials prior 
to their use in more conventional off-line purposes within Oil. The 
argument put forth is tliat this should provide faster development of 
materials as well as more detailed formative evaluational documentation 
and substantiation. If this is really the case, this tyPfi of conputer 
utilization could become a highly inportant function of^e Navy CMI 
system and should "not be considered suppl^nental or si:q3poTttive in 
nature. Curriculum creation and especially revisions is an extremely 
costly process. If computer based developments Qould facilitate both 
the timing and cost aspects , this would be a major breakthrough in the 
DOD training world. ^ • 

9.3 Conclusions 

1. Given the RSD *and extensive^ successful ^ growth of the Navy 
CMI system, the further oihancement by proven researdi prototypes and 
findings should be pursued so as to broaden the impact of individualized 
training , 

2. The selection and j jnplementatian of RgD prototypes should be 
based on rigorous criteria a nd the h igh likelihood of enliancing the 

cost effect iveness of the systeii. * ^ 

^ " — — ■ ■ • ■ ' 

3. The considered elements of instructioilal strategies and 
Instructional System Develo pment activities appear to be the most likely 
candidates for inplementation > 
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QiAPTCR 10 . Summary of the Navy CMI Study 

The Navy CMI Systan represents the most, outstanding large computer 
basqid individualized instructional system to date. There are many 
teasdhs for thi?" outstanding I^avy achievement. First and foremost, 
the training effectiveness found within this system has been and 
continues to be exenplary: The logistic achievemmtlof supporting in 
excess of 3,000 students represents a first in this field. More 
drasnatically , the cost beneficial outtomes yielding a savings- Qf $10,2 
million during FY 75 are rarely found in. the beginning life cycle of 
a training system. Finally , the Navy CMI system has provided for 
effective institutional integration so that its implementation has 
patterned into the common practices and styles of Navy technical 
training while achievingJ its oWn unique benefits. 

The MSU study team utilized a number of different methods in the 
preparation of this monograph. First, documents, both past and current 
were utilized to develop a general framework, ^bst importantly, inter- 
views vdth significant designers and Navy managers were held (without 
the support of all of the Navy a^;id ci^vilian personnel, this monograph 
could never have been drawn together) . Survey techniques were utilized 
in gathering the data concerning the leami^ig center stpenrisors and 
the ISD personnel. "Finally, the CMI system provided computer analysis 
vniich documented its own effectiveness . Such computer analysis con- 
ceming its operation is considered a sine qua non for any computer 
based training system. 

The development and performance data yielded to date by the Nayy 
CMI system provides its strongest quantitative argument for supporting 



133 




the expansion of its: outstanding capability. First,, the CMI system 
has yielded significant course reductions ranging from. 24 to, 80 
percent with a mean of 46,8 percent. This; in^turn, has yielded a 
$10,1 million saving in student salaries. Second, the effective reduc- 
tim m the number of on-board students has allowed for an associated 
reduction of 23 pei\:ent in instructional/support perso 
)^ields a savings of $1.7 million. Third, the CMI training approach, 
yields sigriificantly better end of course performance levels while 
the attitudes of students tend to be more positive. The cost bene- 
ficial effects of this factor are yet to be precisely determined. 
In turn, the CMI system tends to significantly lower the attrition 
rate to between 4.5 to 11.1 percent in 'magnitude. This yields approxi- 
mately $550,000 in savings for FY 75. Finally, the cou$)uter inplemen- 
tation, both as currently operating and in terms of the acquisition of 
the Honeywell system for expansion, has yielded significant saving? 
both in terms of the- conpetitive procurement as well as the potential 
for expanded capability. For these quantitative reasons the Navy CMI 
system can be judged to be the most effective, large-scale, conputer 
based ttaihing system to date. . 

In terms of the -institutional training processes, the CMI system 
haf effectively integrated itself within both personnel and operational 
procedures. For exairple, the learning center supervisors as well as 
ISD personnel perceive CMI as the best approach to individualizing the 
training process. Each of these groups are highly committed to its 
successful inplementation and operation. This is theri reflected in 
the high positive attitude of students toward both their learning 
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center supervisors and 0^ . Given the high involvement o£ the Navy 
unifom personnel in this inplementation effort,. all of the personnel 
perceive the challenge that QH presents and recognize the advantages 
of the syston, its effective time savings and instructional objectives. 
In conq^arison to other conputer bases instructional systems , involved , 

personnel have significantly better attitudes . 

* » 

As noted in the early sections, of the monograph, the Navy system 
also represents an outstanding exampJe ^ f how R^D activity culminates 
in fruition of an actual training operation. The research climate, 
shared civilian, and uniformed personnel, a commitment to sound training 

o 

design, and an adaptive approach t6 CMI systems goals undoubtedly 
allowed it to move from £he R^D phase to fiill operation in less ^^i^ 
a decade. • In comparison with the University of Illinois PLATO system 
and the Air Force Advanced Instructional System, Navy CMI represents 
the largest of these approaches and seems a natural candidate for both 
its planned expansion and its further deployment in the future. VMle 
each of these three computer based systems has its own unique purposes, 
goals , and implementation characteristics , which leads one to recommend 
their continuance,- the Navy QSQ "system -is yielding performance and 
cost benefits that are especially attractiw during the mid '70's. 

The future of the Navy CMI system is already designed. It shall 
grow to support 17,000 students by 1980. During the course of this 
study, new fenhancements have been identified. Classifying these under 
approaches representiiig^instructional strategies and instructional 
systems development activities, a sensible re-infiision of proven R§D 
prototypes would be highly cost beneficial during the coming years. 
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Thus , in ^ sense , this monograph ends vdiere it begins ; that is , in a 
realization that ^research and development can again contribute to this 
outstanding system \Mch fortunately has been designed to infuse new 
ideas an41concepts virile maintaining its high cost beneficial inpact. 
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Review of CMI Projects 



The develbpment of conputer managed instructional syst^ began 
in approximately 196^ (Molnar and Sheiman, *1969) . The development of 
these systems requires adequate support over an extended perijj^ of 
time ; therefore , they are still in an evolutionar;^ stage , The develop - 
ment is also^^-dependent on the "state of the art" iii^each of the many 
caI^x>nents vdiich mist be integrated within the CML system. For exanple, 
the advances in computer technology have been such that systems must 
be continually evaluated and revised. TMsiis particularly^ tnie since 
many of these advances are providing additior^ capabilities at. lower 
costs Some of the significant ccmiputer developments have been \ 
reported by MITRE Corporation (Stetten, Morton, and ^^ayer, 1970)Y 

Subsequent sections of this review will focus on (1) conceptuali- \ 
zation$ of CMI, (2) five large scale projects vrfxich are representative 
of developments in CMI within the civilian educational sector, arid 
(3) military applications of QH. The military \projects are not- 
^describeci in detail in this section as the develoranent of the largest 
of these, Navy CMI, is developmentally and summatavely represented in 
the present rqport as well as in a more« intensive Experimental assess-, 
ment of the project (Carson et al. , 1975). In turn, the Advanced 
Instructional System of the Air Force is described in Chapter S of ^ 
this report. While there are a number of other significant military 
CMI pro^cts (e.g. , MAS,' Lamoore; MAS , Miramar; USASCS , Ft. Monmouth ; 



NCAS, Twentynine Palms j etc.), their key features are siibsumed under 
prior sections of this document, 

1.0 Conceptualizations of CMI ^ 

Although various investigators have offered somewhat differing 
descriptions of GNU, there is a consensus of conceptions that CMI is 
inextricably involved vdth the individualization of instruction. ^ 
Specifically, (ML is viewed as a methodology for realizing the poten- 
tial of computers in supporting individualization of the educatiopatT 
process, - ' 

Glaser (1969)^ offers a general instructional model vAiich he presents 
as a sequence of six operations, all of'vJiich could be subsumed under 
the tending, "individualized instruction/' Though this general mpdel 
can be implemented along a continuum of varying degrees of automation, 
it is suggested that "autonation can be a significant aid to the conduct 
of ah individualized system and to the collection of research data so ^ 
that the system can be improved (Cooley 5 Glaser, 1969)/' For these 
authors, QAI is conceptualized as a means of iinpleirenting those opera- 
tions vAiich characterize an individualized instructional model. 

Still within an individualized framework, Jerman (1970) offe.rs a 
slightly different formulation from tliat discussed above. This formu- 
lation regards CMI — computer mediated instruction-- as being closely 
related to the concept of multimedia instruction. The most ijipDrtant 
.distinction between the two concepts is that in computer-mediated 
instruction, the sequence of topics must be under con5)uter control , 
even \*ien the student is off-line. The function of this system as 
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visualized by Jerman is to w ^ topic, (2) test, (3) pre- 

scribe appropriate offr line materials for study, (4) review the o££- 
liiie-^rtiterial , (5) retest, '(6) report to the teacher, and (7) introduce 
a new topic, etc. 

Peiiiaps the most inclusive conceptualizations regarding CMI have 
been fomulated by co -investigators at Florida State University. Dick 
(1969) describes CMI as: 

' The overall management ofi learning materials and evaluation of 
the students who are participating in a training program vJiere . 
the instruction is not primarily conducted by the pomputer, but 
through the use of oQTer types of self-instructional materials. 

Sen (1970) includes the major educational functions of (1) iiiforma- 
retrieval, (2) scieTatific computing, and (3) con5)uter si^jport of 
instruction within the broader concept of an information management 
system. The last of these major functions, viz., computer si4>port of 
instruction, encompasses CM, conputer- assisted instruction (CAI), and 
learning simulations. Further, it is Hansen's contention tKat CMI 
offers the most^^psC^Ffei^ive. model as well as the greatest potential 
for subsuming the other two types, i.e., CAI and learning simulations 
(Hagerty, 1970). ^ » 

The most significant aspect of FSU*s approach to CMI is that the 
majority of the diagnostic evaluations and remedial prescriptions occur 
withiri a computer teiminal -oriented interaction between the student and 
the CMI system. This technique insures real-time information exchange 
between the student and the system as well as immediately providing 
the student with his next learning assignment. Thus, the interactive 
terminal is the vehicle by vdiich the individualization process under 
is inplemented. 
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The Navy definition of CMI is found in OPNAVINST 1500.39: "A 
system in vMch, a computer is used to route a trainee through a series 
of ijistrurtional mterials , presented by various media , so as to be 
best suited to. his particular needs" and abilities." The instniction 
also, defines key wyrds and terns such as 4rainee,'' "instructional 
materiai," etc. , in such a way that the.co^lete definition is contained ^ 
within the instruction . (TAEG , 1974).- 

2.0 rivi lian CMI Projects | 

Pro2ect_PLAN. Project PLAN (Program for Learning^ in Accordance 

with Ne^) was originated m 1966 through a joint venture involving 
the American Institutes- for Research ,.the Westinghouse Learning Corpo- 
ration, and twelve school districts throughout the country (Flanagan, 
1970-) '. It has now spread to 24 cities involving 20 ,000 students 

. (Rogers, 1971)." * - . . " . 

The nroposed, function of the new educational program is to . 
' nrovid^a flexible syftm in which the student can be assisted 
t^SSe as much resp^ibility as possible in the j.lanning and 
carrying out of his own educational development (Flanagan, 
1970, p. -2) . 

The five major components of the PLAN system are: (a) a set of 
educational objectives; (b) learning methods and materials; (c) evalua- 
tion; (d) guidance and individual planning; and (e) teacher development 

(Flanagan, 1970) . ,« 

The role of the coi^uter in PLAN involves a great variety of 
functions. The computer "p-cesses the daily tests taken by the students 
and provides print-outs of these results for the teachers. These data 
are used to constantly update and revise the student's program of 



<4 ■ ! 



ERIC 



r- 141 

Studies. A vreekly]. status report is also provided. In addition to the, 
daily and weekly processing functions, the computer assists in regis- 
tration of the student and in the actual planning of his course of 
study , including the placement of the student in the program of studies , 
estabUshings^t/<fuota in terms of numbers of modules to be conpleted, 
and a selection of the actual modules of instruction to be iteceived by 
the student ^(Flaiiagan, 1970). The Constant i5)dating and revision of 
student data and ,the planning and prescriptive usage are critical ' 
features in making this type of computer application practical. In 
an effort to facilitate the flow 'of infomation to ahd^pm the computer 
each of the Project PLAN schools has recently received a terminal 
through vMch the teachers can interact directly with the computer 
(Rogers, 1971). 

System Development Corporation/Southwest Regional Laboratory . 
Working wtside the realm of individualized instruction but designed 
to assist teachers in a traditional elementary school setting achieve 
a measure of individualization, a CMI effort was originated under the 
joint sponsorship of the S]^-stem Development Corporation and the South- 
west Regional Laboratory for Educational Research and Development 
(Silberman, 1969) This CMI system was designed to tielp the teacher 
monitor the progress of the students and make decisions as to the pace 
of instruction, the grptping of children, the sequence of lessohs, and 
the individualization qf instruction,)_The four primary components of 
the information management system are objectives, tests, repjorts, and 
prescriptions. It helps teachers by providing information about each 
child's acMeyement, suggesting activities to help a pupil understand 
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♦ a lesson, and Rk)vidii)g a framework for making .decisions on ^laissroQm 
managanent^ (feddes 5,J(ooi, ^969).^ 

IndiAddu^ly Prescribed Instruction . Perhaps the most far -ranging 
effort in the CMI field was the Individually. Prescribed Instruction 
(IPI) pro jW/ instituted in the Oakleaf School System in Pittsburgh by 
the Learning Research and Development Center of the University of 
Pittsburgh. '^The IPI program is based on an instructional model con- 
sisting of the following sequence of operations: 

♦ 1. The goals of learning are specified in terms of observable 

student behavior and the conditions under vrfiich this btehavior 
is .to be exercised, 
2. Diagnosis is tM^^^-^^the imtial capabilities with which the 
learner begins a particular course of instruction. The capa- 
bij-ities that are assessed are those relevant to the forth- 
leoming instruction. 
J 3. Educational alternatives .adaptive' to the initial profile of 

the student^re pres^ented to him • The student selects or is 
assigned oneS^f thesV-jaltematives . 

4. Studjent perfoiiHance is7 monitored and continuously assessed as ' 
the student proceeds to^ learn, 

5. Instruction proqeeds as a function of the relationship between 
measures of student perfbimance, available instructional 
alternatives., and criteriY of competence. 

6. As instruction proceeds* data are generated for monitoring 
and inproAdng the instructional system (Cooley Glaser, ^ 

. 1969 , p. 96) . 0 « ^ 

Automation is not a prerequisite for the -inplementation of 
\ . IPX irodel, and the project initially operated in the nonautomated form, 
\ much as it is being used in scliool systems across the country at 
present (see RERS Reports) . After three years of operation in the 

i 

nonautomated foiin, ba^ch- process computer capabilities were added to 

the program in the form of a Management Information System (MS) . 

There are four major functions wliich the MIS can provide in an 
individualized school: (1) collect data; (2) monitor student 
progress; (3) provide prescriptions; and (4) diagnose student diffi-« 
culty. These functions have two primary objectives: to increase 
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the effectiveness of the model for individualizing instruction 
and to maximize the** productivity of the teacher operating the 
IPI system (Cooley § Glaiser^ 1969 , p. 106) . 

TTirough supporting the IPI instructional system with the functions 



of the MIS, IPI/MIS has b^p shown to be one of tlie leading projects 



T 

in the research and ur^ lament at ion of individualized con^uter- based 



instruction. Gooley and Glaser (1969) admit that a shortcoming of t^je 
IPI/MIS systen is that each school has only one teminal, and it i^'not 
in the classroom: They spe^il ate that the next step in the development 
of IPI/MIS is to add classroom terminal capability so that both students 
and teachers will have access to con^uter terminals. The paper on 
computer-based testing by Ferguson (1970) , an associate of Cooley and 
Glaser, indicates that IPI/MIS is moving in that direction at the 
present time. 

. Thus WB can see that IPI/MIS is unique in that it is the only 
project which has existed in a nonautomated form, has moved to batch- 
process CMI , and is now moving into the other QfE mode , vrfiich was earlier 
described as terminal -oriented Oil. 



Instructiq^ial^ Management System (IMS) 

Coulson (1969) reports that the primary purposes of the computer- 
assisted instructional management (CAIM) system are to help the teacher 
monitor student performance and make management decisions about meeting 
each student *s instructionkl needs. The instructional process is not 
managed by computer but by a teacher who is aided Ijy information prtndded 
by a computer. The computer is not used as a teaching device and does 
not corammicate directly with the student in any way. 
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The initial INK effort at the first grade level provided the 
fol lowing , functions-. 

1. Cairse objectives were -specifically defined and measured by 
multiple choice tests v^^iich were optically scanned. 

2. ^ The directions for takii^^the test were contained on audio 
tapes arid were administered on an individual basis through headsets. 

3. Tests were collected twice each day and were automatically 
evaiuated against mastery levels for one or more behavioral objectives. 

'H. The teacher received computer prescriptive print-outs for each 
student the following morning at the latest., 

5. S^piary reports were provided periodically and/or, upon request. 

Fimctions are also being developed \>Mch will provide computer;:, 
generated exercises where existing materials were found to be inadequate • 

Dr. Coulson noted that there are several advantages of the Instruc- 
tional Management System over tutorial Conputer-Assisted Instruction 
(CAI) . Existing resources can be utilized more effectively, and 
neither major reorganization of the classroom nor large quantities of 
new equipnent or instructional material are required. CAIM can be 
iinplenplnted mach more speedily tlian CAI in most schools. Where indi- 
vidualized study mterials are used, the feedback to the student may 
compare favorably with that provided by CAI , CAIM is designed with 
the teachl&r as '^ts hub , thereby p<?sing les^ of a threat to the teacher 
than CAI, The most obvious advantage of CAIM over CAI is its cost. 
General estimates indicate ^\at CAI is ten to fifty times more expen- 
sive than CAlM. 

The three minimum requirements of a legitimate CAIM system are 
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defined as: 

1. The system must neasure student performance at relatively 
short intervals • 

2. Evaluation should be tied to a set of specified learning 
objectives which are defined in behavioral teiTns. 

3. Specific remedial action should be recomrended to the teacher. 

Florida State Ihiversity's Conputer Managed Instruction Project 

According to Hansen et al. (1973), thrpughout the duration of the 
Themis/CNR contract there was a concurrent set of investigations in 
computer-assisted instruction (CAT) and computer -mana^d instruction 
(CMI) . The primary purpose of the investigations of (M was to deter- 
mine its training effectiveness and associated cost benefit outccgne in 
comparison with CAI and other more conventional means of instruction. 
As pursued at FSU, computer-managed instruction involves the following; 

1. diagnostic assessment and the assignment of individualized 
learning prescriptions, 

2. the vise of CAI for practice and remedial purposes, 

3. the use of simulation fof role and decision -making training 
purposes , 

4. the use of the computer^for ease and objectivity of curriculum 
jievelopment, and 

5. the development of a record system so that the individualized 
trainiilg process could be effectively monitored and managed. 

Within this CMI conceptual context, a number of studies were 
pursued. All of these Studies indicated that M at the collegiate 
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level is highly feasible, cost effective, and gjbvided for learning 
results similar to CAI. However, due to the mastery level learning 
approach utilized in the instructional materials, the relationship of 
individual difference variables to learning' rate or performance was / 
more limited. Where, extensive media and recitation sections v^re used, 
the effects of individual- difference variables seaned to be more pro- 
nounced. Finally, learning attitude toward the instructional materials 
was quite positive and could be manipulated by the form of training. 

In turn, investigations of CAI indicated that it is useful in a 
number of technical training areas. It: proved especially useful for 
dynamic graphics such as found in engineering dynamics. However, while 
CAI was shown to be viable in areas like chemistry, the results did 
not tend to exceed those found in 'OH . 

It is important to note the fact that the development process for 
CMI, while not quite as demanding as that of CAI, still was considerable 
The dependency on a sound training model , formative evaluation , and 
effective monitoring of students in an individualized mode seem to be 
the critical factors in the design and ijrplementation of Q1I. 

From this, consequently , the following research generalizatidns 
can be derived: 

1. Tenninal -oriented computer-managed instrjiction has been shown 
to be urare effective than conventional instruction and less costly than 
computer-assisted instjxictioi^ 

2. The nrast sigilificant gains in the quality of instruction, have 
not necessarily been due to the use of canputers , but have been through 
the iiiplementation of systematic approaches to the training process 
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, required for application o£ the computer. 

3* Although the computer provides the instructional developer 
with more infoxmatibn about the instructional process than has b^n 
available, th6 revision process remains the least well understood and 
utilized conponent of the systems approach; however, the provision for 
systematic, reliable data now allows us to turn our attention to this 
'problem. 

4. Interdisciplinary collegiate development teams will not 
necessarily produce better computerized instructional materials than ^ 
those produced by conceptually integrated teams. 

3.0 Military^ Applications of CMI ^ v 

Fletcher et al. (1975) reports that there is a variety of research 
development, and inplementation effort^ in the three services v*iich 
include CMI. The Army Conputeiri zed Training System (CTS) and Air Force 
Advanced Instructional System (AIS) will have CMI capabilities. How- 
ever, these systems may be limited in their Oil applications because 
they are designed to permit rapid response times to student inputs, 
and they may provide insufficient file support for some types of C^^I. 
The Computer-Assisted Instruction Study Management System (CAISM5) 
(Uhiversit/ of Illinois) also inclucles both CAI and CMI. 

A six -month analytical survey and study was conducted by the 
Training Analysis and Evaluation Group (TAEG) of the Naval Training 
Equipment Center. The principal conclusions of the study (Middleton, 
Papetti, and Micheli, 1974) are quoted below: 
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*'a. There is no alternative for the Navy but to go to CMI 
i£ any significant number of its over 4,000 courses are to becon» 
self -paced and individualized (\>Mch is the trend of current ^ 
educational technology) (p , 63) 

"b. Preliminary tradeoff analyses made during this investi- 
gation reveal that a comMnation of minicomputers (strategically 
located to, perform the routjyie tasks of OH) and a ceiltral 
computer system (for high level management information processing) 
is irare cost effective th&n a single large-scale centralized 
computer (p. 63) ^ 

''c. A minicomputer for small, remote^zlasses Vs feasible... • 
it is proposed that CMI training in renme sites be linked 
together via land lines. In this concept, a greater number of 
managers and students can utilize the capabilities of (M and 
have a ntore -cost-effective system.'' (p. 64) 

^'d. The use of shipboard tactical computers for managing 
individual training has long been desired by the training commu- 
nity. However, numerous technical and logistical problems, as 
well as priorities placed iqx)n the use of shipboard conputers by 
higher authorities , have allowed relatively little training via 
operational computers aboard ships .... the state-of-the-art is 
advancing at such a pace in the mini- and micro- computer field 
that in. the near future the market price for these systems will 
be such that it will be economically irore advantageous for ships 
to have a dedicated^ sys tern for education, rather than iirplen^nting 
a retrofit 'program to use operational equipment and computers for 
(p. 64-5) 
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APPENDIX B 

MEMPHIS STATE UNIVERSITY 
COMPUTER-MANAGED INSTRUCTIOfcl SURVEY 
INSTRUCTOR QUESTIONNAIRE 

i^The purpose of this survey is to obtain the professional judgment of 
instructors cx^nc^ming the Navy's CMI program. The information from the 
survey will be used ta^^cribe Ins^tructors ' perceptions concerning CMI 
and to determine needa* inprovements • The results will provide the basis 
for further planning and substantiation of the Navy's Oil program. 

^ /^Please respond to the items ^ accurately and honestly as you can. 

^1 responses will be confidential; you are not asked to identify yourself 
on the questionnaire^ Hopefully, the items on the questiojinaire are clearly 
stated. If you have difficulty with any of the questions , you ran contact 
Dr. Clyde Smith (Base, ext. 5429 ; Outside, 872-4976). 

Thank you for your assistance. 



Part I. Deroogr^hic Data 



,1. Age 



a. Less than 21 

b. - 21 - 25 

c. 26 - 30 



d. 31 - 35 

e. 36 - 40 

f . over 4D' 



2. Years in military 



a. Less than 2 years 

b. 2-5 years 



c. 6 - 10 years 

d. more than 10 years 



3. Status (Present) 



c. 



Enlisted 

Warrant 

Commissioned 



d. Civil Service 

e. Other 



4. Years e:q)erience with conputers 

a. Less than 1 

b. 1 - 3 

c. 4 - 6 



d. 7 - 9 

e. 10 or more 



ERLC 



ERIC 
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5. Present educational status 

a. did not finish high school d. college diploma 

b. high school diploma/G.E.D. e. advanced degree 

c. some college experience 

... 

Part II. Opinionnaire ' 

Check only one ansj/er for each question, unless otherwise instructed. 

1. tn comparison to all of the training. uiN^ich your service is currently 
engaged, indicate the extent to ivhich you think it is using computers 
to assist in tlie instructional process.^ 



a. 


0 - 


5% 


d. 


21 - 30% -. 


b. 


6 - 


10% 


e . 


31 - 50% 


c. 


11 


- 20% 


f. 


greater than 50% 



2. With regard to your answer to question Ifl, to v*iat extent 

computers should be used to assist in instruction? . ^^"^^ 

a. much TiK>re than now ^ d. slightly less than now 

b. 'slightly mote than now^ e. much less than now 

c. about the same as now ^ 



3. Projecting to 10 years from now, how do you think the extent of 
computer usage in instruction mil conparp to today^ usage? 

a. much more d. slightly less 

b. slightly m>T^ e. much less 

c. about the same 



To what extent do you agree or disagree vdth this statement: "As far 
as, the military is concerned, the feasibility and effectiveness of 
CMI are no longer in question. Our goal now is to find the best ways 
of implerrenting computerized instruction.*' 

a. strongly agree d. disagree 

b, agree e. strongly disagree 



c. no opinion 



n 

J. o 
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What do you perceive to be the two most significant problems (or 
barriers) relating to the present use of computer .technology in 
militaiy training? (Check two) 

a; cost , N d. 'insufficient hardware 

technology ' 

^ ^ b. instructor attitudes - e. organizational clira^e 

' - c. student attitudes f. other (specify) 



Given a choice, viiich one of the following instructional modes would 
you prefer to use for tHe courses in vMch you presently are involved? 

• . ^ ■ " 

a. conventional mode - d. programmed manuals \ 

classrooms, blackboards, 
^textbooks, etc. 

b. Q1I ^ e. other (specify) 

c. tutorial GAI , 



What type of student doi you feel benefits most from conputerized 
instruction? ( 

a. above average in learning ability 



b. average "in Ite^ming ability 

c. below average in learning ability 

d. all benefit about the same 



How would you evaluate the influence on learning outcori«s of instructors 
in CMI con5)ared to the influence of conventional classroom teachers? 

a. considerably more influential d. slightly less 

influential > 

\ b. slightly more influential ^ e, considerably less 

infliiehtial 

c. about the same 

Indicate your reaction t© the following statement: 

■-■V ■ ■ ■ ^ 

* Military training today* is still very labor intensive. In the future, 
the augmentation of instructors by machines , such as computers , offers 
the most likely altemktive to increase productivity in graining." 

J a. strongly agree d. disagree - 



jb. agree ■ e. strongly disagree 

c. not sufe 



\ . - 
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10. Indicate your reaction to the following statement: 

*'In the future , motivation for enlisting in the service could be 
increased as a result of modem training methods, such as computer- 
based instruction/' 

^^^^^ a. strongly agree ^ d, disagree 

" b. agree e. strongly disagree 

c. not sure 



11. In your opinion, which one of the following activities requires the 
most time and effort from the learning supervisor? 

a. basic administrative responsibilities 

^ b. answering questions from students ' 

c. providing ren^dial assistance / 



dv counseling students 

e. disciplining students 

f. other (specify) 



12, What are the top three benefits or advantages of CNtl? (Choose 3) 

^ a. It saves tiine. 

b. It saves money. « ' 

c. It produces quality instruction. 

d. It is favored by students. 

e. It has the flexibility to handle varying training loads.: 

f. It reduces the dropout rate. 

^ g. It offers greater uniforfnity in the quality of training. 

- h. . Itvproyides greater assurance that educational objectives 



vdll be met. 



i. It is adaptable to individual differences, 
j . Other (specify) . - 
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13. What two categqries of personnel are likely to benefit most from CMI? 
(Choose 2) 

^ ■ 

a. enlisted men learning entry level skills 

b. enlisted men beyond the entry level 

c. cadets 



■ .,■ - ■ 
\ d. officer eaitty^dates 

e. officers at the basic or^vanced level 

f . other (specify) . 



14. Recent official surveys indicate that: "Compared to the draftee, 
today^s typical enlistee has a lower educational level, is sevtfral 
years younger, and tends to sign for a unit of choice which allows 
units home on weekends." Therefore, training by CMI will be more 
^ appropriate than conventional methods. 

a. strongly agree d. disagree 

^^b. agree e. strongly disagree 

c. not sure ^ 



15. Select one or more ways in vMch you think instructors , working with 
- conputer based training systems, should be selected for assignment. 

a. according to prior knowledge of conputers 

b. according to prior teaching ejqperience 

c* because they volunteer 



^ d. according to M>S^ AFSC/NEC, or sub-s{)ecialty code or 
skill identities 

e. according to personality and attitude test scores 



^ f. without too much enphasis on any of the above, but more 
according to availability 



Part III . Open-ended Questions 



Please try to make your answers brief a?id concise; if possible use a 
few key words or short sentences. ; 

1. How iiiqx)rtant is the role of the instructor in a CMI system? 



•i. I • f> 



ERIC 
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£• How successful was on-the-job training in preparing you for OH? 



3. How useful wLs Instructor Training School in preparing you for your 
CMI functions »in the area of: 

a. instructional techniques 



instructional naterials 
.c. testing 

d. Instructional System Developnent 



4. What proportion of the students in your CMI program show some evidence 
that they are not really trying to succeed? 



5. What techniques do you use to motivate students v*ien using CM' 



6. Overall, how successful has CMI irplementation been in your training 
program? 



7. Has CMI increased the productivity of your training program? 

/ 

8. How successful is the CMI system in achieving instructional objectives 
of your training program? 



9. What are the most successful features of the CMI system as it operates 
in your training program? 
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10* VVhat are the least successful features of the CM system as it 
operates in your training program? 



11. If you have any additional coiranents (a word, a phrase, a sentence, 
or.ijiore) about CMI, please write them below: 




